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Statistics Basic concepts of probability theory 

2 Basic concepts of probability theory 

2.1 Probability space, probability function, sample space, event 

A probability space is a pair (fi, P) consisting of a set Q and a function P which assigns to each 
subset A of £1 a real number P(A) in the interval [0, 1]. Moreover, the following two axioms are 
required to hold: 

l.P(ft) = 1, 

2. P (U^Li ^) = J2™=i P(An) if Ai, A2, . . . is a sequence of pairwise disjoint subsets of $1. 
The set fi is called a sample space. The elements 00 E S~i are called sample points and the subsets 
A C fi are called events. The function P is called a probability function. For an event A, the 
real number P(A) is called the probability of A. 

From the two axioms the following consequences can be deduced: 

3. P(0) = 0, 

4. P(A\B) = P(A) - P(B) if P C A, 

5. P(CA) = 1 - P(A), 
6.P(A) ^ P(B) if B a, 

7. P(Ai U • • • U An) = P(Ai) H h P(A n ) if Ai, . . . , A n are pairwise disjoint events, 

8. P{A UB) = P(A) + P(P) - P(A n P) for arbitrary events A and P. 

Example. Consider the set £1 — {1, 2, 3, 4, 5, 6}. For each subset A of fi, define 

where #A is the number of elements in A. Then the pair (ft, P) is a probability space. One can 
view this probability space as a model for the for the situation "throw of a dice". 

Example. Now consider the set £1 = {1, 2, 3, 4, 5, 6} x {1, 2, 3, 4, 5, 6}. For each subset A of Q, 
define 

p(^> = *± . 

V ; 36 
Now the probability space (fl, P) is a model for the situation "throw of two dice". The subset 

A = {(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)} 

is the event "a pair". 

2.2 Conditional probability 

For two events A and B the conditional probability of A given B is defined as 

P(AnB) 



P(A | B) := 



P(B) 
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We have the following theorem called computation of probability by division into possible causes: 
Suppose A\, . . . , A n are pairwise disjoint events with A\ U • • • U A n = Q. For every event B it 
then holds that 

P(B) = P(Ai) • P(B | A 1 ) + • • • + P{A n ) - P(B | A n ) . 



Example. In the French Open final, Nadal plays the winner of the semifinal between Federer 
and Davydenko. A bookmaker estimates that the probability of Federer winning the semifinal is 
75%. The probability that Nadal can beat Federer is estimated to be 51%, whereas the probability 
that Nadal can beat Davydenko is estimated to be 80%. The bookmaker therefore computes the 
probability that Nadal wins the French Open, using division into possible causes, as follows: 



P (Nadal wins the final) 



P (Federer wins the semifinal) x 

P (Nadal wins the final | Federer wins the semifinal) + 

P (Davydenko wins the semifinal) x 

P (Nadal wins the final | Davydenko wins the semifinal) 

0.75-0.51 + 0.25-0.8 

58.25% 
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Statistics Basic concepts of probability theory 

2.3 Independent events 

Two events A and B are called independent, if 

P(A nfl) = P{A) • P{B) . 

Equivalent to this is the condition P(A \ B) = P(A), i.e. that the probability of A is the same as 
the conditional probability of A given B. 



Remember: Two events are independent if the probability of one of them is not affected by 
knowing whether the other has occurred or not. 



Example. A red and a black dice are thrown. Consider the events 

A: red dice shows 6, 
B: black dice show 6. 

Since 

P ^ nB) = 36 = 6-6= P( - 4) - P(B) ' 
A and B are independent. The probability that the red dice shows 6 is not affected by knowing 

anything about the black dice. 

Example. A red and a black dice are thrown. Consider the events 

A: the red and the black dice show the same number, 
B: the red and the black dice show a total of 10. 

Since 

P(A) = -, but P(A | B) = - , 

A and B are not independent. The probability of two of a kind increases if one knows that the sum 
of the dice is 10. 

2.4 The Inclusion-Exclusion Formula 

Formula 8 on page 12 has the following generalization to three events A,B,C\ 

P(AuBuC) = P(A) + P(B) + P(C)-P(AnB)-P(AnC)-P(BnC) + P(AnBnC). 
This equality is called the Inclusion-Exclusion Formula for three events. 

Example. What is the probability of having at least one 6 in three throws with a dice? Let A\ be 
the event that we get a 6 in the first throw, and define A2 and As similarly. Then, our probability 
can be computed by inclusion-exclusion: 

p = P(A 1 UA 2 UA 3 ) 

= P(A X ) + P{A 2 ) + P(A 3 ) - P{A r n A 2 ) - P{A r n A 3 ) - P(A 2 n As) 
+P{Ai nA 2 n As) 



6 6 6 


" 6 2 " 


" 6 2 " 


" 6 2 + 6 3 


41% 
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The following generalization holds for n events Ai, A2, . . . , A n with union A = A\ U • • • U A n \ 

p{A) = J2p(A) ~J2 p ( A * n A i) + J2 p ( Ai n a j n A k) - ' ' • ± p (^i n • • • n A n ) . 

i i<j i<j<k 

This equality is called the Inclusion-Exclusion Formula for n events. 

Example. Pick five cards at random from an ordinary pack of cards. We wish to compute the 
probability P{B) of the event B that all four suits appear among the 5 chosen cards. 

For this purpose, let A\ be the event that none of the chosen cards are spades. Define A 2l As, 
and A4 similarly for hearts, diamonds, and clubs, respectively. Then 

ZB = A x U A 2 U A 3 U A 4 . 

The Inclusion-Exclusion Formula now yields 

p{Zb) = J2 p i A i) - J2 p ( A * n A i) + J2 p ( A * n A i n A k) ~ p ( A ± n ^ni 3 n A 4 ) , 

i i<j i<j<k 

that is 

t 3 ;) ( 2 ;) (?) 

P(ZB) = 4 • -) (- - 6 • -) f + 4 • -) (- - « 73.6% 

(?) (?) (?) 

We thus obtain the probability 

P(B) = 1 - P(CB) = 26.4% 



Example. A school class contains n children. The teacher asks all the children to stand up and 
then sit down again on a random chair. Let us compute the probability P(B) of the event B that 
each pupil ends up on a new chair. 

We start by enumerating the pupils from 1 to n. For each i we define the event 

Ai : pupil number i gets his or her old chair 

Then 

ZB = A x U • • • U A n . 

Now P(ZB) can be computed by the Inclusion-Exclusion Formula for n events: 

P(Cfl) = Y^P{Ai) ~Y.n A i nA i) + • • • ± p (^i n • • • n AO > 

thus 
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We conclude 



P{B) = 1 - P(ZB) 



111 1 

± — 



2! 3! 4! n\ 

It is a surprising fact that this probability is more or less independent of n: P(B) is very close to 
37% for all n > 4. 



2.5 Binomial coefficients 



n 



The binomial coefficient I (read as "n over fc") is defined as 



n 



n\ 



l-2-3---n 



kj k\(n-k)\ l-2---k-l-2---(n-k) 

for integers n and k with ^ k ^ n. (Recall the convention 0! = 1.) 

The reason why binomial coefficients appear again and again in probability theory is the fol- 
lowing theorem: 



The number of ways of choosing k elements from a set ofn elements is 



n 




m 



rfe 



It's only an 
opportunity if 

you act on it 
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For example, the number of subsets with 5 elements (poker hands) of a set with 52 elements (a 

pack of cards) is equal to 

'52 N 

An easy way of remembering the binomial coefficients is by arranging them in Pascal's tri- 
angle where each number is equal to the sum of the numbers immediately above: 



2598960 . 




1 

1 1 

121 

133 1 

14641 

15 10 10 5 1 

1 6 15 20 15 6 1 



One notices the rule 



n 
n — k 



n 



,e.g. 



2.6 Multinomial coefficients 

The multinomial coefficients are defined as 

/ n 



n\ 



...kJ 



\ k\ • • • k r I k\\- • • k 

for integers n and fei, . . . , k r with n = k\ -\ \- k r . The multinomial coefficients are also called 

generalized binomial coefficients since the binomial coefficient 



is equal to the multinomial coefficient 



with I — n — k. 



n 
k I 
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3 Random variables 

3.1 Random variables, definition 

Consider a probability space (fi, P). A random variable is a map X from Q into the set of real 
numbers R. 




Normally, one can forget about the probability space and simply think of the following rule of 
thumb: 



Remember: A random variable is a function taking different values with different probabilities. 



The probability that the random variable X takes certain values is written in the following way: 

P(X = x): the probability that X takes the value xEl, 
P(X < x): the probability that X takes a value smaller than x, 
P(X > x)\ the probability that X takes a value greater than x, 
etc. 



One has the following rules: 

P(X < x) 

P(X > x) 



1 



P{X < x) + P{X = x) 
P(X >x) + P(X = x) 
P(X <x) + P(X = x) + P(X > x) 



3.2 The distribution function 

The distribution function of a random variable X is the function F : R — ► R given by 

F(x) = P(X < x) . 

F{x) is an increasing function with values in the interval [0, 1] and moreover satisfies F(x) — > 1 
for x —> oo, and F(x) —> for x —> — oc. 

By means of F(x), all probabilities of X can be computed: 

P(X < x) = lim £ ^o F(x - e) 

P(X = x) = F{x) - lim^o F(x - e) 

P{X >x) = 1 - lim £ ^o F(x - e) 

P(X >x) = 1 - F(x) 
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3.3 Discrete random variables, point probabilities 

A random variable X is called discrete if it takes only finitely many or countably many values. 
For all practical purposes, we may define a discrete random variable as a random variable taking 
only values in the set {0, 1, 2, ... }. The point probabilities 

P(X = k) 

determine the distribution of X. Indeed, 

P(i eJ 4)^P(I = fc) 
keA 

for any AC {0, 1,2,...}. In particular we have the rules 

P(x<k) = EtoP(x = i) 



P(X>k) = Y?= k P(x = i) 

The point probabilities can be graphically illustrated by means of a pin diagram: 
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P(x=k) 










0,2 














0,1 


















2 3 < 


\ 5 


6 


7 





3.4 Continuous random variables, density function 

A random variable X is called continuous if it has a density function f{x). The density function, 
usually referred to simply as the density, satisfies 



P(X e A) = [ f{t)dt 

JteA 



for all A C R. If A is an interval [a, b] we thus have 



P(a < X < b) = f f{t)dt . 

J a 



One should think of the density as the continuous analogue of the point probability function in the 
discrete case. 



3.5 Continuous random variables, distribution function 

For a continuous random variable X with density f(x) the distribution function F(x) is given by 

F(x) = f f(t)dt . 

J — OO 

The distribution function satisfies the following rules: 

P(X <x) = F(x) 

P(X >x) = 1 - Fix) 

P(\X\<x) = F(x)-F(-x) 

P(\X\>x) = F(-x) + l-F(x) 

3.6 Independent random variables 

Two random variables X and Y are called independent if the events iGi and Y e B are in- 
dependent for any subsets i,BCK. Independence of three or more random variables is defined 
similarly 



Remember: X and Y are independent if nothing can be deduced about the value of Y from 
knowing the value of X. 
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Example. Throw a red dice and a black dice and consider the random variables 

X: number of pips of red dice, 

Y: number of pips of black dice, 

Z: number of pips of red and black dice in total. 

X and Y are independent since we can deduce nothing about X by knowing Y. In contrast, X 
and Z are not independent since information about Z yields information about X (if, for example, 
Z has the value 10, then X necessarily has one of the values 4, 5 and 6). 

3.7 Random vector, simultaneous density, and distribution function 

If Xl, . . . , X n are random variables defined on the same probability space (fi, P) we call X = 
(Xl, . . . , X n ) an (n-dimensional) random vector. It is a map 

X : Q -► R n . 

The simultaneous (n-dimensional) distribution function is the function F : W 1 —> [0, 1] given by 

F(xi, . . . , x n ) = P(X 1 < xi A • • • A X n < x n ) . 

Suppose now that the X{ are continuous. Then X has a simultaneous (n-dimensional) density 
f :R n ^ [0,oo [satisfying 

P(X e A) = f f (x) dx 

for all ACR n . The individual densities fi of the Xi are called marginal densities, and we obtain 
them from the simultaneous density by the formula 

fi{xi)= I f(xi,...,x n )dx2--.dx n 

JR 71 - 1 

stated here for the case fi(xi). 



Remember: The marginal densities are obtained from the simultaneous density by "integrating 
away the superfluous variables". 



4 Expected value and variance 

4.1 Expected value of random variables 

The expected value of a discrete random variable X is defined as 

oo 

E(X) = 5^P(X = fc)-fc. 
fc=i 

The expected value of a continuous random variable X with density f(x) is defined as 

/oo 
f(x) -xdx. 
-OO 

Often, one uses the Greek letter ji ("mu") to denote the expected value. 
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4.2 Variance and standard deviation of random variables 

The variance of a random variable X with expected value E(X) = ji is defined as 

var(X) = E((X - /i) 2 ) . 

If X is discrete, the variance can be computed thus: 

oo 

var(X) = Y^ p ( x = k)-(k- /i) 2 . 
If X is continuous with density f(x), the variance can be computed thus: 

/oo 
f(x)(x — \i) 2 dx . 
-oo 

The standard deviation a ("sigma") of a random variable X is the square root of the variance: 

a(X) = ^var(X) . 
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4.3 Example (computation of expected value, variance, and standard deviation) 

Example 1. Define the discrete random variable X as the number of pips shown by a certain 
dice. The point probabilities are P(X = k) = 1/6 for k = 1, 2, 3, 4, 5, 6. Therefore, the expected 
value is 

E(X) = ± I. k = 1+1+3+1+5 + 6 = 35 



fc=l 



6 



6 



The variance is 



var(X) = ±Uk- SXf = d - 3-5)' + (2 - 3.5)* + - .. + (6 - 3.5)' = ^ 



fe=l 



The standard deviation thus becomes 

a(X) = V2.917= 1.708. 



Example 2. Define the continuous random variable X as a random real number in the interval 
[0, 1]. X then has the density f(x) = 1 on [0, 1]. The expected value is 



The variance is 



The standard deviation is 



E(X) = / xdx = 0.5. 
Jo 

(X) = [ (x-0.5) 2 dx = 0.083. 
Jo 

a = V0.083 = 0.289 . 



4.4 Estimation of expected value /i and standard deviation a by eye 

If the density function (or a pin diagram showing the point probabilities) of a random variable is 
given, one can estimate /i and a by eye. The expected value fi is approximately the "centre of 
mass" of the distribution, and the standard deviation a has a size such that more or less two thirds 
of the "probability mass" lie in the interval ji ± a. 



i 


i 






(x) 








0,2 
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4.5 Addition and multiplication formulae for expected value and variance 

Let X and Y be random variables. Then one has the formulae 

E(X + Y) = E(X) + E(Y) 

E(aX) = a-E(X) 

var(X) = E(X 2 ) - E(X) 2 

var(aX) = a 2 • var(X) 

var(X + a) = var(X) 

for every a G M. If X and Y are independent, one has moreover 

E{X • Y) = E{X) • E(Y) 

var(X + y) = var(X) + var(y) 



Remember: The expected value is additive. For independent random variables, the expected value 
is multiplicative and the variance is additive. 



4.6 Covariance and correlation coefficient 

The covariance of two random variables X and Y is the number 

Cov(X, Y) = E((X - EX)(Y - EY)) . 

One has 

Cov(X,X) = var(X) 

Cov(X, Y) = E(X Y)-EXEY 

var(X + Y) = var(X) + var(y) + 2 • Cov(X, Y) 

The correlation coefficient p ("rho") of X and Y is the number 

Cov(x,y) 

9 ~ a{X) • a{Y) ' 



where cr(X) = ^/var(X) and <r(Y) = ^/var(y) are the standard deviations of X and Y. It is 
here assumed that neither standard deviation is zero. The correlation coefficient is a number in the 
interval [— 1, 1]. If X and Y are independent, both the covariance and p equal zero. 



Remember: A positive correlation coefficient implies that normally X is large when Y large, and 
vice versa. A negative correlation coefficient implies that normally X is small when Y is large, 
and vice versa. 



Example. A red and a black dice are thrown. Consider the random variables 

X: number of pips of red dice, 

Y: number of pips of red and black dice in total. 
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If X is large, Y will normally be large too, and vice versa. We therefore expect a positive correla- 
tion coefficient. More precisely, we compute 

E(X) = 3.5 

E(Y) = 7 

E(X ■ Y) = 27.42 

a(X) = 1.71 

a{Y) = 2.42 

The covariance thus becomes 

Cov(X, Y) = E{X ■ Y) - E(X) ■ E(Y) = 27.42 - 3.5 • 7 = 2.92 . 

As expected, the correlation coefficient is a positive number: 

Cov(X, Y) 2.92 



a{X) ■ a(Y) " 1.71 • 2.42 



0.71 
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5 The Law of Large Numbers 

5.1 Chebyshev's Inequality 

For a random variable X with expected value ji and variance a 2 , we have Chebyshev's Inequal- 
ity: 

2 

P{\X-ii\ >a)< ^ 
for every a > 0. 

5.2 The Law of Large Numbers 

Consider a sequence Xi, X<i, X$, . . . of independent random variables with the same distribution 
and let \i be the common expected value. Denote by S n the sums 

S n = X\ + • • • + X n . 

The Law of Large Numbers then states that 



P 



n 
for every e > 0. Expressed in words: 






> e ) —> for n —> oc 



The mean value of a sample from any given distribution converges to the expected value of that 
distribution when the size n of the sample approaches oc. 



5.3 The Central Limit Theorem 

Consider a sequence X\ , X2 , X3 , . . . of independent random variables with the same distribution. 
Let \i be the common expected value and a 2 the common variance. It is assumed that a 2 is positive. 
Denote by S' n the normed sums 

Qf _ Xi-\ h Xn-flfJi 

a^Jn 

By "normed" we understand that the S f n have expected value and variance 1. The Central Limit 
Theorem now states that 

P(S' n < x) — > $(x) for n — ► oc 

for all xEl, where $ is the distribution function of the standard normal distribution (see section 
15.4): 



$(x) = / -= e"2 t dt . 



The distribution function of the normed sums S r n thus converges to $ when n converges to oc. 

This is a quite amazing result and the absolute climax of probability theory! The surprising 
thing is that the limit distribution of the normed sums is independent of the distribution of the Xi. 
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5.4 Example (distribution functions converge to $) 

Consider a sequence of independent random variables X\ , X2 , . . . all having the same point prob- 
abilities 

P(X X = 0) = P{X X = 1) = -. 



The sums S n = X\ + • • • + X n are binomially distributed with expected value fi = n/2 and 
variance a 2 = n/4. The normed sums thus become 

Xi + • • • + X n - /i/2 



o^ 



V^/2 



The distribution of the S' n is given by the distribution function F n . The Central Limit Theorem 
states that F n converges to $ for n — ► 00. The figure below shows F n together with $ for n = 
1, 2, 10, 100. It is a moment of extraordinary beauty when one watches the F n slowly approaching 
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6 Descriptive statistics 

6.1 Median and quartiles 

Suppose we have n observations x\ , . . . , x n . We then define the median ir (0.5) of the observations 
as the "middle observation". More precisely, 



x(0.5) 



£( n +i)/2 if n is odd 

v (x n/2 + x n/2+1 )/2 if n is even 

where the observations have been sorted according to size as 

X\ < X 2 < • • • < X n . 
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Similarly, the lower quartile #(0.25) is defined such that 25% of the observations lie below 
#(0.25), and the upper quartile #(0.75) is defined such that 75% of the observations lie below 
#(0.75). 

The interquartile range is the distance between x(0.25) and #(0.75), i.e. #(0.75) — #(0.25). 



6.2 Mean value 

Suppose we have n observations x\ , . . . , x n . We define the mean or mean value of the observa- 
tions as 

n 

6.3 Empirical variance and empirical standard deviation 

Suppose we have n observations #i, . . . , x n . We define the empirical variance of the observa- 
tions as 

S ~ n-l 




www.im^rith-zf 
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The empirical standard deviation is the square root of the empirical variance: 



V n-l 

The greater the empirical standard deviation s is, the more "dispersed" the observations are around 
the mean value x. 

6.4 Empirical covariance and empirical correlation coefficient 

Suppose we have n pairs of observations (xi, yi), . . . , (x n , y n ). We define the empirical covari- 
ance of these pairs as 

_ T,?=i(xi-x)(yi-y) 



Cov, 



emp — -. 

71 — 1 



Alternatively, Cov emp can be computed as 



Cov, 



emp — ., 

n — 1 



The empirical correlation coefficient is 

empirical covariance Cov emp 



r — 



(empirical standard deviation of the x)(empirical standard deviation of the y) s x s y 
The empirical correlation coefficient r always lies in the interval [—1,1]. 



Understanding of the empirical correlation coefficient. If the x-observations are independent of 
the ^-observations, then r will be equal or close to 0. If the x-observations and the ^/-observations 
are dependent in such a way that large x-values usually correspond to large y-values, and vice 
versa, then r will be equal or close to 1. If the x-observations and the ^/-observations are dependent 
in such a way that large x-values usually correspond to small ^/-values, and vice versa, then r will 
be equal or close to -1. 



7 Statistical hypothesis testing 

7.1 Null hypothesis and alternative hypothesis 

A statistical test is a procedure that leads to either acceptance or rejection of a null hypothesis 
Hq given in advance. Sometimes Hq is tested against an explicit alternative hypothesis Hi. 

At the base of the test lie one or more observations. The null hypothesis (and the alternative 
hypothesis, if any) concern the question which distribution these observations were taken from. 

7.2 Significance probability and significance level 

One computes the significance probability P, that is the probability - if Hq is true - of obtaining 
an observation which is as extreme, or more extreme, than the one given. The smaller P is, the 
less plausible Hq is. 
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Often, one chooses a significance level a in advance, typically a = 5%. One then rejects Hq 
if P is smaller than a (and one says, "Hq is rejected at significance level a"). If P is greater than 
a, then Ho is accepted (and one says, "Ho is accepted at significance level a" or "Ho cannot be 
rejected at significance level a"). 

7.3 Errors of type I and II 

We speak about a type I error if we reject a true null hypothesis. If the significance level is a, 
then the risk of a type I error is at most a. 

We speak about a type II error if we accept a false null hypothesis. 

The strength of a test is the probability of rejecting a false Hq. The greater the strength, the 
smaller the risk of a type II error. Thus, the strength should be as great as possible. 

7.4 Example 

Suppose we wish to investigate whether a certain dice is fair. By "fair" we here only understand 
that the probability p of a six is 1/6. We test the null hypothesis 

Ho : p = - (the dice is fair) 
6 

against the alternative hypothesis 

Hi : p > - (the dice is biased) 
6 

The observations on which the test is carried out are the following ten throws of the dice: 

2,6,3,6,5,2,6,6,4,6. 

Let us in advance agree upon a significance level a = 5%. Now the significance probability P 
can be computed. By "extreme observations" is understood that there are many sixes. Thus, P is 
the probability of having at least five sixes in 10 throws with a fair dice. We compute 



10 /m\ 

p = J2[k) (V 6 )*( 5 /6) 10 "* = °-° 15 

L._C \ / 



k=5 

(see section 8 on the binomial distribution). Since P = 1.5% is smaller than a = 5%, we reject 
Hq. If the same test was performed with a fair dice, the probability of committing a type I error 
would be 1.5%. 

8 The binomial distribution Bin(n, p) 

8.1 Parameters 

n: number of tries 

p: probability of success 

In the formulae we also use the "probability of failure" q = 1 — p. 
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8.2 Description 

We carry out n independent tries that each result in either success or failure. In each try the 
probability of success is the same, p. Consequently, the total number of successes X is binomially 
distributed, and we write X ~ Bin(n, p). X is a discrete random variable and takes values in the 
set {0, 1, . . . , n}. 



8.3 Point probabilities 

For k E {0, 1, ... , n}, the point probabilities in a Bin(n, p) distribution are 



P(X = k) 



k jn—k 



•p -q 



See section 2.5 regarding the binomial coefficients 



n 
k 



Example. If a dice is thrown twenty times, the total number of sixes, X, will be binomially 
distributed with parameters n = 20 and p = 1/6. We can list the point probabilities P(X = k) 
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and the cumulative probabilities P(X > k) in a table (expressed as percentages): 



k 





12 3 


4 


5 


6 


7 8 9 


P(X = k) 
P(X > k) 


2.6 
100 


10.4 19.8 23.8 
97.4 87.0 67.1 


20.2 
43.3 


12.9 
23.1 


6.5 
10.2 


2.6 0.8 0.2 

3.7 1.1 0.3 



8.4 Expected value and variance 

Expected value: E(X) = np. 
Variance: var(X) = npq. 

8.5 Significance probabilities for tests in the binomial distribution 

We perform n independent experiments with the same probability of success p and count the 
number k of successes. We wish to test the null hypothesis Hq : p = po against an alternative 
hypothesis Hi. 

Ho Hi Significance probability 

P = Po P > Po P(X > k) 
P = Po P < Po P(X < k) 
P = Po P + Po E/ p ( x = 
where in the last line we sum over all / for which P{X — I) < P{X — k). 

Example. A company buys a machine that produces microchips. The manufacturer of the ma- 
chine claims that at most one sixth of the produced chips will be defective. The first day the 
machine produces 20 chips of which 6 are defective. Can the company reject the manufacturer's 
claim on this background? 

SOLUTION. We test the null hypothesis Hq : p = 1/6 against the alternative hypothesis Hi : 
p > 1/6. The significance probability can be computed as P(X > 6) = 10.2% (see e.g. the table 
in section 8.3). We conclude that the company cannot reject the manufacturer's claim at the 5% 
level. 

8.6 The normal approximation to the binomial distribution 

If the parameter n (the number of tries) is large, a binomially distributed random variable X 
will be approximately normally distributed with expected value /i = np and standard deviation 
a = ^npq. Therefore, the point probabilities are approximately 

i'k — np\ 1 

P{X = k)^(p l J: ] 



y/npq J y/npq 

where ip is the density of the standard normal distribution, and the tail probabilities are approxi- 
mately 



P{X < k) w $ 



k + \ 



np 



^rvpq 
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P(X > k) w 1 - $ 



k — \ — np 



y/npq I 
where $ is the distribution function of the standard normal distribution (Table B.2). 



Rule of thumb. One may use the normal approximation if np and nq are both greater than 5. 



Example (continuation of the example in section 8.5). After 2 weeks the machine has produced 
200 chips of which 46 are defective. Can the company now reject the manufacturer's claim that 
the probability of defects is at most one sixth? 

SOLUTION. Again we test the null hypothesis Hq : p = 1/6 against the alternative hypothesis 
Hi : p > 1/6. Since now np « 33 and nq « 167 are both greater than 5, we may use the normal 
approximation in order to compute the significance probability: 

(46 — - — 33 3 \ 
2 ' « 1 - $(2.3) « 1.1% 
V27.8 y 

Therefore, the company may now reject the manufacturer's claim at the 5% level. 

8.7 Estimators 

Suppose k is an observation from a random variable X ~ Bin(n,p) with known n and unknown 
£>. The maximum likelihood estimate (ML estimate) of p is 

. k 

P= ~ • 
n 

This estimator is unbiased (i.e. the expected value of the estimator is p) and has variance 

var(p) = — . 
n 

The expression for the variance is of no great practical value since it depends on the true (un- 
known) probability parameter p. If, however, one plugs in the estimated value p in place of p, one 
gets the estimated variance 

P(l-P) 
n 

Example. We consider again the example with the machine that has produced twenty microchips 
of which the six are defective. What is the maximum likelihood estimate of the probability param- 
eter? What is the estimated variance? 

Solution. The maximum likelihood estimate is 

and the variance of p is estimated as 

0.3 -(1-0.3) 



20 



0.0105 
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The standard deviation is thus estimated to be V0.0105 « 0.10. If we presume that p lies within 
two standard deviations from p, we may conclude that p is between 10% and 50%. 



8.8 Confidence intervals 

Suppose k is an observation from a binomially distributed random variable X ~ Bin(n,p) with 
known n and unknown p. The confidence interval with confidence level 1 — a around the point 
estimate p = k/n is 



; p(i-p) ., /p(i-p) 

P-^l-a/2\ , P + ^l-a/2^ 



n 



n 



Loosely speaking, the true value p lies in the confidence interval with the probability 1 — a. 

The number ui_ a / 2 is determined by $(^1-^/2) = 1 — a /2 where $ is the distribution 
function of the standard normal distribution. It appears e.g. from Table B.2 that with confidence 
level 95% one has 

u l-a/2 = ^0.975 = 1-96 . 

Exercise. In an opinion poll from the year 2015, 62 out of 100 persons answer that they intend 
to vote for the Green Party at the next election. Compute the confidence interval with confidence 
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level 95% around the true percentage of Green Party voters. 

Solution. The point estimate is p = 62/100 = 0.62. A confidence level of 95% yields a = 
0.05. Looking up in the table (see above) gives i/o.975 = 1-96. We get 



, _ /0.62-0.38 _ 

196 V-I5S- = - 10 ' 

The confidence interval thus becomes 

[0.52 , 0.72] . 

So we can say with a certainty of 95% that between 52% and 72% of the electorate will vote for 
the Green Party at the next election. 

9 The Poisson distribution Pois(A) 

9.1 Parameters 

A: Intensity 

9.2 Description 

Certain events are said to occur spontaneously, i.e. they occur at random times, independently 
of each other, but with a certain constant intensity A. The intensity is the average number of 
spontaneous events per time interval. The number of spontaneous events X in any given concrete 
time interval is then Poisson distributed, and we write X ~ Pois(A). X is a discrete random 
variable and takes values in the set {0, 1, 2, 3, . . . }. 

9.3 Point probabilities 

For k E {0, 1, 2, 3 . . . } the point probabilities in a Pois(A) distribution are 

P(X = fe) = -exp(-A). 
Recall the convention 0! = 1. 

Example. In a certain shop an average of three customers per minute enter. The number of 
customers X entering during any particular minute is then Poisson distributed with intensity A = 
3. The point probabilities (as percentages) can be listed in a table as follows: 



k 





1 


2 


3 


4 


5 


6 


7 


8 


9 


> 10 


P(X = k) 


5.0 


14.9 


22.4 


22.4 


16.8 


10.1 


5.0 


2.2 


0.8 


0.3 


0.1 



9.4 Expected value and variance 

Expected value: E{X) = A. 
Variance: var(X) = A. 
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9.5 Addition formula 

Suppose that Xi, . . . , X n are independent Poisson distributed random variables. Let A^ be the 
intensity of X{, i.e. Xi ~ Pois(A^). Then the sum 

X = X 1 + • • • + X n 
will be Poisson distributed with intensity 

A = Ai + • • • + A n , 
i.e. X - Pois(A). 

9.6 Significance probabilities for tests in the Poisson distribution 

Suppose that k is an observation from a Pois(A) distribution with unknown intensity A. We wish 
to test the null hypothesis Ho : A = Ao against an alternative hypothesis Hi. 

Hq Hi Significance probability 



A = A A > A P(X > k) 
A = A A < A P(X < k) 
A = A A^A £jP(A: = 

where the summation in the last line is over all / for which P{X — I) < P(X = k). 

If n independent observations fei, . . . , k n from a Pois(A) distribution are given, we can treat 
the sum k = k\ + • • • + k n as an observation from a Pois(n • A) distribution. 

9.7 Example (significant increase in sale of Skodas) 

Exercise. A Skoda car salesman sells on average 3.5 cars per month. The month after a radio 
campaign for Skoda, seven cars are sold. Is this a significant increase? 

SOLUTION. The sale of cars in the given month may be assumed to be Poisson distributed with a 
certain intensity A. We test the null hypothesis 

H : A = 3.5 

against the alternative hypothesis 

Hi : A > 3.5 . 

The significance probability, i.e. the probability of selling at least seven cars given that Ho is true, 

is 

°° (3 5) k 
P = ^2 fri ex P(~ 3 - 5 ) = °- 039 + °- 017 + °- 007 + °- 002 + • • • = 0.065 . 

k=7 
Since P is greater than 5%, we cannot reject Hq. In other words, the increase is not significant. 
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9.8 The binomial approximation to the Poisson distribution 

The Poisson distribution with intensity A is the limit distribution of the binomial distribution with 
parameters n and p = X/n when n tends to oc. In other words, the point probabilities satisfy 

P(X n = *;)-► P(X = k) for n ^ oc 

for X rsj Pois(A) and X n ~ Bin(n, X/n). In real life, however, one almost always prefers to use 
the normal approximation instead (see the next section). 



9.9 The normal approximation to the Poisson distribution 

If the intensity A is large, a Poisson distributed random variable X will to a good approximation 
be normally distributed with expected value \i — X and standard deviation a = VX The point 
probabilities therefore are 

p(x = k)^v n ''~ Xx l 



Vx ) Vx 

where tp(x) is the density of the standard normal distribution, and the tail probabilities are 



P(X < k) w $ 



Vx 
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P ( X> t)Kl _*(^i^j 

where $ is the distribution function of the standard normal distribution (Table B.2). 



Rule of thumb. The normal approximation to the Poisson distribution applies if A is greater than 
nine. 



9.10 Example (significant decrease in number of complaints) 

Exercise. The ferry Deutschland between R0dby and Puttgarten receives an average of 180 
complaints per week. In the week immediately after the ferry's cafeteria was closed, only 112 
complaints are received. Is this a significant decrease? 

Solution. The number of complaints within the given week may be assumed to be Poisson 
distributed with a certain intensity A. We test the null hypothesis 

H : A = 180 

against the alternative hypothesis 

Hi : A < 180 . 

The significance probability, i.e. the probability of having at most 112 complaints given Ho, can 
be approximated by the normal distribution: 

/ll2+i-180\ 

P = $ ^= = $(-5.03) < 0.0001 . 

^ VT80 J 

Since P is very small, we can clearly reject Hq. The number of complaints has significantly 
decreased. 

9.11 Estimators 

Suppose fei, . . . k n are independent observations from a random variable X ~ Pois(A) with un- 
known intensity A. The maximum likelihood estimate (ML estimate) of A is 

A = (fci H h k n )/n . 

This estimator is unbiased (i.e. the expected value of the estimator is A) and has variance 

var(A) = — . 
n 

More precisely, we have 

nX ~ Pois(nA) . 

If we plug in the estimated value A in A's place, we get the estimated variance 

var(A) = — . 
n 
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9.12 Confidence intervals 

Suppose k\ , . . . , k n are independent observations from a Poisson distributed random variable X ~ 
Pois(A) with unknown A. The confidence interval with confidence level 1 — a around the point 
estimate A = (fei H \- k n )/n is 



Loosely speaking, the true value A lies in the confidence interval with probability 1 — a. 

The number Ui_ a / 2 is determined by &(ui_ a / 2 ) = 1 — olJ% where $ is the distribution 
function of the standard normal distribution. It appears from, say, Table B.2 that 

u l-a/2 = ^0.975 = 1-96 

for confidence level 95%. 

Example (continuation of the example in section 9.10). In the first week after the closure of the 
ferry's cafeteria, a total of 112 complaints were received. We consider k = 112 as an observation 
from a Pois(A) distribution and wish to find the confidence interval with confidence level 95% 
around the estimate 

A = 112. 

Looking up in the table gives ^0.975 = 1-96. The confidence interval thus becomes 
112 - 1.96VTl2 , 112 + 1.96VTl2 1 « [91 , 133] 

10 The geometrical distribution Geo(p) 

10.1 Parameters 

p: probability of success 

In the formulae we also use the "probability of failure" q = 1 — p. 

10.2 Description 

A series of experiments are carried out, each of which results in either success or failure. The 
probability of success p is the same in each experiment. The number W of failures before the first 
success is then geometrically distributed, and we write W ~ Geo(p). W is a discrete random 
variable and takes values in the set {0, 1,2,...}. The "wait until success" is V = W + 1. 

10.3 Point probabilities and tail probabilities 

For k G {0, 1, 2 . . . } the point probabilities in a Geo(p) distribution are 

P(X = k) = q k p . 
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In contrast to most other distributions, we can easily compute the tail probabilities in the geomet- 
rical distribution: 

P{X >k) = q k . 

Example. Pin diagram for the point probabilities in a geometrical distribution with probability 
of success p = 0.5: 
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Statistics The hypergeometrical distribution HG(n,r,N) 

10.4 Expected value and variance 

Expected value: E(W) = q/p. 
Variance: var(VF) = q/p 2 . 

Regarding the "wait until success" V = W + 1, we have the following useful rule: 



Rule. The expected wait until success is the reciprocal probability of success: E(V) = 1/p. 



Example. A gambler plays lotto each week. The probability of winning in lotto, i.e. the proba- 
bility of guessing correctly seven numbers picked randomly from a pool of 36 numbers, is 

/36\ _1 
p= I j ^0.0000001198. 

The expected wait until success is thus 

E(V) = p' 1 = ( J weeks = 160532 years . 

11 The hypergeometrical distribution HG(n, r, N) 

11.1 Parameters 

r : number of red balls 

s : number of black balls 

N: total number of balls (N = r + s) 

n: number of balls picked out (n < N) 

11.2 Description 

In an urn we have r red balls and s black balls, in total N = r + s balls. We now pick out 
n balls from the urn, randomly and without returning the chosen balls to the urn. Necessarily 
n < N . The number of red balls Y amongst the balls chosen is then hypergeometrically dis- 
tributed and we write Y ~ HG(n, r, N). Y is a discrete random variable with values in the set 
{0, 1, . . . ,min{n,r}}. 

11.3 Point probabilities and tail probabilities 

For k G {0, 1, . . . , min{n, r}} the point probabilities of a HG(n, r, N) distribution are 



P(Y = k) 



k 1 \ n — k 

"TTT 
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Example. A city council has 25 members of which 13 are Conservatives. A cabinet is formed by 
picking five council members at random. What is the probability that the Conservatives will have 
absolute majority in the cabinet? 

Solution. We have here a hypergeometrically distributed random variable Y ~ HG(5, 13, 25) 
and have to compute P(Y > 3). Let us first compute all point probabilities (as percentages): 



k 





1 


2 


3 


4 


5 


P(Y = k) 


1.5 


12.1 


32.3 


35.5 


16.1 


2.4 



The sought-after probability thus becomes 

P(Y > 3) = 35.5% + 16.1% + 2.4% 



54. 



11.4 Expected value and variance 

Expected value: E{Y) = nr/N. 

Variance: var(y) = nrs(N - n)/(N 2 (N - 1)). 

11.5 The binomial approximation to the hypergeometrical distribution 

If the number of balls picked out, n, is small compared to both the number of red balls r and the 
number of black balls s, it becomes irrelevant whether the balls picked out are returned to the urn 
or not. We can thus approximate the hypergeometrical distribution by the binomial distribution: 

P(Y = k) « P(X = k) 

for Y ~ HG(n, r, N) and X ~ Bin(n, r/N). In practice, this approximation is of little value 
since it is as difficult to compute P(X = k) as P(Y = k). 

11.6 The normal approximation to the hypergeometrical distribution 

If n is small compared to both r and s, the hypergeometrical distribution can be approximated by 
the normal distribution with the same expected value and variance. 
The point probabilities thus become 



/ 



P(Y = k)&<p 



k — nr/N 



\ 



1 



K ^Jnrs(N-n)/(N 2 (N-l))J \]nrs{N - n)/(N 2 (N - 1)) 



where ip is the density of the standard normal distribution. The tail probabilities become 

/ . , ... \ 

P{Y <k)^$ 



k+\ 



nr/N 



y ^nrs(N - n)/(N 2 (N - 1)) j 
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/ 



P(Y > k) ps 1 - $ 



/€ 77 



nr/N 



\ 



^nrs(N - n)/(N 2 (N - l))y 
where $ is the distribution function of the standard normal distribution (Table B.2). 



12 The multinomial distribution Mult(n, pi , 

12.1 Parameters 

n: number of tries 

pi : 1 st probability parameter 



>Pr) 



p r : rth probability parameter 

It is required that pi + • • • + p r = 1. 

12.2 Description 

We carry out n independent experiments each of which results in one out of r possible outcomes. 
The probability of obtaining an outcome of type i is the same in each experiment, namely pi. Let 
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Si denote the total number of outcomes of type i. The random vector S = (Si, . . . , S r ) is then 
multinomially distributed and we write S ~ Mult(n,pi, . . . ,p r ). S is discrete and takes values in 

the set {(fci, . . . k r ) G 27 | /^ > , fci + • • • + k r = n}. 

12.3 Point probabilities 

For k\ + • • • + k r = n the point probabilities of a Mult(n,pi, . . . ,p r ) distribution are 



P(S = (k 1 ,...,k r ))=( " )Hp 1 



2=1 

Example. Throw a dice six times and, for each z, let Si be the total number of z's. Then 
S = (Si, . . . , Se) is a multinomially distributed random vector: S ~ Mult(6, 1/6, . . . , 1/6). 
The probability of obtaining, say, exactly one 1, two 2s, and three sixes is 

P(S = (1, 2, 0, 0, 0, 3)) = ( i 2 o 6 Q Q 3 J • (1/6) 1 • (1/6) 2 • (1/6) 3 » 0.13% 

Here, the multinomial coefficient (see also section 2.6) is computed as 

f 6 \ 6! 720 ^ n 

60 . 



12 0003 / 1!2!0!0!0!3! 12 



12.4 Estimators 



Suppose fei, . . . , k r is an observation from a random vector S ~ Mult(n,pi, . . . ,p r ) with known 
n and unknown p^. The maximum likelihood estimate of pi is 

Pi = — • 
n 

This estimator is unbiased (i.e. the estimator's expected value is pi) and has variance 

P*(l-P*) 



var(^ 



n 



13 The negative binomial distribution NB(n, p) 

13.1 Parameters 

n: number of tries 

p: probability of success 

In the formulae we also use the letter q = 1 — p. 
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13.2 Description 

A series of independent experiments are carried out each of which results in either success or 
failure. The probability of success p is the same in each experiment. The total number X of failures 
before the n'th success is then negatively binomially distributed and we write X ~ NB(n, p). The 
random variable X is discrete and takes values in the set {0, 1, 2, ... }. 

The geometrical distribution is the special case n = 1 of the negative binomial distribution. 

13.3 Point probabilities 

For k E {0, 1, 2 . . . } the point probabilities of a NB(fc, p) distribution are 

p(x=k)=( n+ n k _- i y P ". q K 

13.4 Expected value and variance 

Expected value: E{X) = nq/p. 
Variance: var(X) = nq/p 2 . 

13.5 Estimators 

The negative binomial distribution is sometimes used as an alternative to the Poisson distribution 
in situations where one wishes to describe a random variable taking values in the set {0,1,2,...}. 
Suppose k\ , . . . , k m are independent observations from a NB(n, p) distribution with unknown 
parameters n and p. We then have the following estimators: 

k 2 A k 

n= — T , p= -J 

s z — k s z 

where k and s 2 are the mean value and empirical variance of the observations. 

14 The exponential distribution Exp (A) 

14.1 Parameters 

A: Intensity 

14.2 Description 

In a situation where events occur spontaneously with the intensity A (and where the number of 
spontaneous events in any given time interval thus is Pois(A) distributed), the wait T between 
two spontaneous events is exponentially distributed and we write T ~ Exp (A). T is a continuous 
random variable taking values in the interval [0, oo[. 
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14.3 Density and distribution function 

The density of the exponential distribution is 

f(x) = A • exp(— Ax) . 

The distribution function is 

F(x) = 1 — exp(— Ax) . 

14.4 Expected value and variance 

Expected value: E(T) = 1/A. 
Variance: var(T) = 1/A 2 . 

15 The normal distribution 

15.1 Parameters 

/i: expected value 

a 2 : variance 

Remember that the standard deviation a is the square root of the variance. 
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15.2 Description 

The normal distribution is a continuous distribution. If a random variable X is normally distribut- 
ed, then X can take any values in R and we write X ~ iV(/i, a 2 ). 

The normal distribution is the most important distribution in all of statistics. Countless natu- 
rally occurring phenomena can be described (or approximated) by means of a normal distribution. 

15.3 Density and distribution function 

The density of the normal distribution is the function 



V2^ ' V 2tj" 
It is symmetric, i.e. 



(t - M ) 2 



K-x) = f{x) . 
The distribution function of the normal distribution 

is difficult to compute. Instead, one uses the formula 

' x — jl 



F(x) = $ 



a 



where $ is the distribution function of the standard normal distribution which can be looked up in 
Table B.2. From the table the following fact appears: 



Fact: In a normal distribution, about 68% of the probability mass lies within one standard devi- 
ation from the expected value, and about 95% of the probability mass lies within two standard 
deviations from the expected value. 



15.4 The standard normal distribution 

A normal distribution with expected value \i = and variance a 2 = 1 is called a standard normal 
distribution. The standard deviation in a standard normal distribution equals 1 (obviously). The 
density ip(t) of a standard normal distribution is 

<P(t) - j= exp {-\r 

The distribution function $ of a standard normal distribution is 
On can look up $ in Table B.2. 



Download free eBooks at bookboon.com 

47 



Statistics The normal distribution 

15.5 Properties of $ 

The distribution function $ of a standard normally distributed random variable X ~ iV(0, 1) 
satisfies 

P(X <x) = <f>(x) 

P(X >x) = $(-x) 

P(\X\<x) = <$>(x) - <£>(-x) 

P(\X\>x) = 2-$(-x) 

<$>(-x) = l-$(x) 

15.6 Estimation of the expected value \i 

Suppose x\, X2, . . . , x n are independent observations of a random variable X ~ N(fi, a 2 ). The 
maximum likelihood estimate (ML estimate) of \i is 

xi-\ h x n 

li= - 



n 



This is simply the mean value and is written x. The mean value is an unbiased estimator of \i (i.e. 
the estimator's expected value is \i). The variance of the mean value is 



2/-\ a2 
var [x) = — 

n 



More precisely, x is itself normally distributed: 



AT/ & \ 

X rsj iVf/i, J . 

n 

15.7 Estimation of the variance a 2 

Suppose a?i, . . . , x n are independent observations of a random variable X ~ iV(/i, a 2 ). Normally, 
the variance a 2 is estimated by the empirical variance 

2 _ Z^\ x i ~ %) 



n — 1 
The empirical variance s 2 is an unbiased estimator of the true variance a 2 . 

Warning: The empirical variance is not the maximum likelihood estimate of a 2 . The maximum 
likelihood estimate of a 2 is 

n 
but this is seldom used since it is biased and usually gives estimates which are too small. 
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15.8 Confidence intervals for the expected value /i 

Suppose x\, . . . , x n are independent observations of a normally distributed random variable X ~ 
7V(/i, a 2 ) and that we wish to estimate the expected value /i. If the variance a 2 is known, the 
confidence interval for n with confidence level 1 — a is as follows: 



(J a 

X-Ui- a / 2 —J= , X J rU 1 _ a/2 



n 



n 



The number Ui_ a / 2 is determined by &(ui_ a / 2 ) = 1 — a/2 where $ is the distribution function 
of the standard normal distribution. It appears from, say, Table B.2 that 

u l-a/2 = ^0.975 = 1-96 

for confidence level 95%. 

If the variance a 2 is unknown, the confidence interval for ji with confidence level 1 — a is 



x-h_ a/2 (n- i)y— , z + *i-a/2(rc-i)v — 



where s 2 is the empirical variance (section 6.3). The number ti_ a / 2 is determined by F(ui_ a / 2 ) = 
1 — a/2, where F is the distribution function of Student's t distribution with n — 1 degrees of 
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freedom. It appears from, say, Table B.4 that 



n 



t 



l-a/2 



6 



9 



10 



11 12 



12.7 4.30 3.18 2.78 2.57 2.45 2.36 2.31 2.26 2.23 2.20 



for confidence level 95%. 

15.9 Confidence intervals for the variance a 2 and the standard deviation a 

Suppose x\ , . . . , x n are independent observations of a normally distributed random variable X 
iV(/i, a 2 ). The confidence interval for the variance a 2 with confidence level 1 — a is: 

(n-l)s 2 (n-l)s 2 



X. 



a/2 



X 



l-a/2 



where s 2 is the empirical variance (section 6.3). The numbers x a /2 anc ^ Xi_ a /2 are determined 
by F(Xa/2) = a /^ anc * F(Xi- a /2) = ^ — OL l^ w ^ ere F is the distribution function of the x 2 
distribution with n — 1 degrees of freedom (Table B.3). 

Confidence intervals for the standard deviation a with confidence level 1 — a are computed 
simply by taking the square root of the limits of the confidence intervals for the variance: 



' (n — l)s 2 (n — l)s 2 



X. 



a/2 



X 



l-a/2 



15.10 Addition formula 

A linear function of a normally distributed random variable is itself normally distributed. If, in 
other words, X ~ N([jl, a 2 ) and a, 6 <E R (a ^ 0), then 

aX + 6- N(afi + 6, a 2 a 2 ) . 

The sum of independent normally distributed random variables is itself normally distributed. 
If, in other words, X\, . . . , X n are independent with X{ ~ iV(/^, a 2 ), then we have the addition 
formula 

X x + • • • + X n - N(fn + • • • + ^ n ,a\ + • • • + a 2 n ) . 

16 Distributions connected with the normal distribution 

16.1 The x 2 distribution 

Let Xl, . . . , X n rsj iV(0, 1) be independent standard normally distributed random variables. The 
distribution of the sum of squares 

is called the x 2 distribution with n degrees of freedom. The number of degrees of freedom is 
commonly symbolized as df. 
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A x 2 distributed random variable Q with df degrees of freedom has expected value 

E(Q) = df 



and variance 



var(Q) = 2 • df . 



The density of the x 2 distribution is 



§L. 



f(x) = K • x 2 1 • e 2 

where df is the number of degrees of freedom and K is a constant. In practice, one doesn't use 
the density, but rather looks up the distribution function in Table B.3. The graph below shows the 
density function with df = 1, 4, 10, 20 degrees of freedom. 




16.2 Student's t distribution 

Let X be a normally distributed random variable with expected value jjl and variance a 2 . Let the 
random variables X and S 2 be the mean value and empirical variance, respectively, of a sample 
consisting of n observations from X. The distribution of 



Vsy 



n 



is then independent of both ji and a 2 and is called Student's t distribution with n — 1 degrees of 
freedom. 

A t distributed random variable T with df degrees of freedom has expected value 



for df > 2, and variance 



E(T) = 
df 



var(T) = 



df-2 



for df > 3. 

The density of the t distribution is 



fto-K.[l + w 



X 



2 v-(#+l)/2 
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where df is the number of degrees of freedom and K is a constant. In practice, one doesn't use 
the density, but rather looks up the distribution function in Table B.4. The graph below shows 
the density of the t distribution with df = 1,2,3 degrees of freedom and additionally the density 
(f(x) of the standard normal distribution. As it appears, the t distribution approaches the standard 
normal distribution when df — > oo. 




16.3 Fisher's F distribution 

Let X\ and X2 be independent normally distributed random variables with the same variance. For 
i = 1, 2 let the random variable Sf be the empirical variance of a sample of size rii from X{. The 
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distribution of the quotient 

s 2 

is called Fisher's F distribution with ni — 1 degrees of freedom in the numerator and n^ — l 
degrees of freedom in the denominator. 
The density of the F distribution is 

x df i/2-l 

f{x) = K ' (df 2 + df^W 2 

where K is a constant, df\ the number of degrees of freedom in the numerator, d/2 the number 
of degrees of freedom in the denominator, and df = df\ + d/2. In practice, one doesn't use the 
density, but rather looks up the distribution function in Table B.5. 

17 Tests in the normal distribution 

17.1 One sample, known variance, H : /x = /x 

Let there be given a sample x\, . . . , x n of n independent observations from a normal distribution 
with unknown expected value /i and known variance a 2 . We wish to test the null hypothesis 

Hq : /i = /io • 
For this purpose, we compute the statistic 

_ y/n(x - fjip) _ Ya=\ x i - n W 

LL I 

a vna 2 

The significance probability now appears from the following table, where $ is the distribution 
function of the standard normal distribution (Table B.2). 

Alternative Significance 
hypothesis probability 



Hi 
Hi 
Hi 



jJL > /iQ $>( — u) 

/i < /iO $(u) 



Normally, we reject Hq if the significance probability is less than 5%. 

17.2 One sample, unknown variance, H : ji = /x (Student's t test) 

Let there be given a sample xi, . . . , x n of n independent observations from a normal distribution 
with unknown expected value /i and unknown variance a 2 . We wish to test the null hypothesis 

Hq : /i = /io • 
For this purpose, we compute the statistic 

Vn(x - /io) _ YJi=i x i ~ n W 



t 



\Jn - s 2 
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where s 2 is the empirical variance (see section 6.3). 

The significance probability now appears from the following table where i^tudent is the dis- 
tribution function of Student's t distribution with df = n — 1 degrees of freedom (Table B.4). 

Alternative Significance 

hypothesis probability 



Hi 
Hi 
Hi 



fl > flo 1- ^Student (*) 

/i < JIQ 1 - Fstudmt(-t) 

fl^flO 2 • (1 - Fstudent(|*|)) 



Normally, we reject Hq if the significance probability is less than 5%. 

Example. The headmaster of a school wishes to confirm statistically that his students have per- 
formed significantly miserably in the 2008 final exams. For this purpose, n = 10 students are 
picked at random. Their final scores are 

X\ X2 Xs X4 X 5 Xq X7 X 8 Xq Xiq 



7.6 7.7 7.5 5.8 5.7 7.9 5.4 6.7 7.9 9.4 

The national average for 2008 is 8.27. It is reasonable to assume that the final scores are normally 
distributed. However, the variance is unknown. Therefore, we apply Student's t test to test the null 
hypothesis 

H : fi = 8.27 

against the alternative hypothesis 

Hi :^<8.27. 

We compute the mean value of the observations as x = 7.17 and the empirical standard deviation 
as s = 1.26. We obtain the statistic 



V10(7.17-8.27) 

t = - — = -2.76 . 

1.26 

Looking up in Table B.4 under df = n — 1 = 9 degrees of freedom gives a significance probability 

1 - ^Student (—*) = 1 _ ^Student (2. 76) 

between 1% and 2.5%. We may therefore reject Ho and confirm the headmaster's assumption that 
his students have performed significantly poorer than the rest of the country. 

17.3 One sample, unknown expected value, H : a 2 = &l 

Theorem. Let there be given n independent observations xi, . . . , x n from a normal distribution 
with variance a 2 . The statistic 



(n-l)s 2 __£?=!(** -* 



i2 



a 2 a 2 



is then x 2 distributed with df = n — 1 degrees of freedom (here s 2 is the empirical variance). 
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Let there be given a sample x\, . . . , x n of n independent observations from a normal distribution 
with unknown expected value ji and unknown variance a 2 . We wish to test the null hypothesis 



H. ^ 2 ^ 2 
o : a — a 



For this purpose, we compute the statistic 



Q = 



(Tn 



(Jn 



where s 2 is the empirical variance. 

The significance probability can now be read from the following table where F x 2 is the distri- 
bution function of the x 2 distribution with df = n — 1 degrees of freedom (Table B.3). 



Alternative 
hypothesis 



Significance 
probability 



Hi 
Hi 
Hi 



CT Z > a z n 



1-F x 2(q) 

F x 2(q) 



a 2 + a 2 2 • min{F x2 (g), 1 - F^q)} 



Normally, Ho is rejected if the significance probability is smaller than 5%. 
Note: In practice, we always test against the alternative hypothesis Hi : a 2 > a\. 
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17.4 Example 

Consider the following twenty observations originating from a normal distribution with unknown 
expected value and variance: 

91 97 98 112 91 97 116 108 108 100 
107 98 92 103 100 99 98 104 104 97 

We wish to test the null hypothesis 

Hq: the standard deviation is at most 5 (i.e. the variance is at most 25) 

against the alternative hypothesis 

Hi: the standard deviation is greater than 5 (i.e. the variance is greater than 25). 

The empirical variance is found to be s 2 = 45.47 and we thus find the statistic 

(20-1) -45.47 nA rn 
q = ± -4 = 34.56 . 

By looking up in Table B.3 under df = 19 degrees of freedom, we find a significance probability 
around 2%. We can thus reject Hq. 

(Actually, the observations came from a normal distribution with expected value // = 100 and 
standard deviation a = 6. The test is thus remarkably sensitive.) 

17.5 Two samples, known variances, H : \i\ — /x 2 

Let there be given a sample x\ , . . . , x n from a normal distribution with unknown expected value 
111 and known variance a\. Let there in addition be given a sample j/i, . . . , y m from a normal 
distribution with unknown expected value \i2 and known variance o\. It is assumed that the two 
samples are independent of each other. 
We wish to test the null hypothesis 

H : jJL\ = /jL2 • 

For this purpose, we compute the statistic 

x-y 



V a i/ n + G ll 



m 



The significance probability is read from the following table where $ is the distribution function 
of the standard normal distribution (Table B.3). 



Alternative 


Significance 


hypothesis 


probability 


Hi : fjii> ii 2 


$(-u) 


Hi : iii < ii 2 


$(u) 


Hi : iii ^ ii 2 


2-$(-H) 


Normally, we reject Hq if the significance probability is smaller than 5% 



Note: In real life, the preconditions of this test are rarely met. 
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17.6 Two samples, unknown variances, H : /ii = /x 2 (Fisher-Behrens) 

Let the situation be as in section 17.5, but suppose that the variances o\ and a\ are unknown. The 
problem of finding a suitable statistic to test the null hypothesis 

H : iii = /i 2 

is called the Fisher-Behrens problem and has no satisfactory solution. 

If n, m > 30, one can re-use the test from section 17.5 with the alternative statistic 

x-y 

u 



\J ' s\jn + s\jm 
where s\ and s\ are the empirical variances of the x's and y's, respectively. 

17.7 Two samples, unknown expected values, H : o\ = o\ 

Let there be given a sample x\ , . . . , x n from a normal distribution with unknown expected value 
111 and unknown variance o\. In addition, let there be given a sample yi, . . . , y m from a normal 
distribution with unknown expected value /12 and unknown variance <j\. It is assumed that the 
two samples are independent of each other. 
We wish to test the null hypothesis 

Ho : g\ = <J2 • 
For this purpose, we compute the statistic 

s\ empirical variance of the x's 
s\ empirical variance of the y's 

Further, put 

/ r 
v — max <v,— 

I v 

The significance probability now appears from the following table where Fpisher is the distribution 
function of Fisher's F distribution with n — 1 degrees of freedom in the numerator and m — 1 
degrees of freedom in the denominator (Table B.5). 

Alternative Significance 

hypothesis probability 



Hi 
Hi 
Hi 



cr( > (J 2 1 - ^Fisher (v) 

<j\<(j\ 1 - ^FisherlV^) 

g\ ^cj 2 2 2-(l- F Fisher (i;*)) 



Normally, Hq is rejected if the significance probability is smaller than 5%. 

If Ho is accepted, the common variance o\ = a\ is estimated by the "pooled" variance 

s 2 = U=i(*i-x) 2 + E?=i(yi-y) 2 = (n - i)sj + (m - 1)4 _ 

po ° n + m — 2 n + TTi — 2 
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17.8 Two samples, unknown common variance, H : /ii = ^2 

Let there be given a sample x\ , . . . , x n from a normal distribution with unknown expected value 
111 and unknown variance a 2 . In addition, let there be given a sample yi, . . . , y m from a normal 
distribution with unknown expected value /12 and the same variance a 2 . It is assumed that the 
two samples are independent of each other. 
We wish to test the null hypothesis 

H : jJLi = fi2 ■ 

For this purpose, we compute the statistic 

x-y 



t 



(l/n+l/m)s 2 00l 



where Sp 0ol is the "pooled" variance as given in section 17.7. 

The significance probability now appears from the following table where i^tudent is the dis- 
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tribution function of Student's t distribution with n + m — 2 degrees of freedom (Table B.4). 



Alternative Significance 

hypothesis probability 



Hi 
Hi 
Hi 



Ml > M2 1 - ^Student (*) 

Ml < M2 1 - ^Student (-*) 

Ml 7^2 2.(l-F S tudent(|t|)) 



Normally, Hq is rejected if the significance probability is smaller than 5%. 

17.9 Example (comparison of two expected values) 

Suppose we are given seven independent observations from a normally distributed random vari- 
able X: 

x\ = 26 , X2 = 21 , xs = 15 , X4 = 7 , x$ = 15 , xq = 28 , £7 = 21 

and also four independent observations from a normally distributed random variable Y: 

yi = 29,y 2 = 31,y 3 = 17,y 4 = 22. 

We wish to test the hypothesis 

H : E{X) = E{Y) . 

In order to be able to test this, we need to test first whether X and Y have the same variance. 
We therefore test the auxiliary hypothesis 

H£ : var(X) = var(y) 

against the alternative 

H^ : var(X) ^ var(y) . 

For this purpose, we compute the statistic 

s? 52.3 



1.26 

S\ 41.(3 

as in section 17.7, as well as 



v* — max <v—> = 1.26 . 

Looking up in Table B.5 with 7—1 = 6 degrees of freedom in the numerator and 4 — 1 = 3 
degrees of freedom in the denominator shows that the significance probability is clearly greater 
than 20%, and we may therefore accept the auxiliary hypothesis Hq. 
Now we return to the test of Ho against the alternative hypothesis 

Hi : E(X) + E{Y) . 

The "pooled" variance is found to be 

2 6s? + 3si 

4ooi = \ 2 = 48.8 . 
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The statistic thereby becomes 

x-y _ 19 - 24.8 

(1/7 + l/4)4~; ~ VW7 + V4)48.8 

Therefore, the significance probability is found to be 

2 • (1 - Fstudent(N)) = 2 • (1 - F stude nt(1.31)) « 2 ■ (1 - 0.90) = 20% 

by looking up Student's t distribution with 7 + 4 — 2 = 9 degrees of freedom in Table B.4. 
Consequently, we cannot reject Hq. 

18 Analysis of variance (ANOVA) 

18.1 Aim and motivation 

Analysis of variance, also known as ANOVA, is a clever method of comparing the mean values 
from more than two samples. Analysis of variance is a natural extension of the tests in the previous 
chapter. 

18.2 k samples, unknown common variance, H : \i\ = • • • = fik 

Let Xi , . . . , Xk be k independent, normally distributed random variables, with expected values 
/ii, . . . , fj,k and common variance a 2 . From each Xi, let there be given a sample consisting of rii 
observations. Let Xj and s 2 be mean value and empirical variance of the sample from Xj. 
We wish to test the null hypothesis 

Ho : /ii = • • • = fJ»k 

against all alternatives. For this purpose, we estimate the common variance a 2 in two different 
ways. 

The variance estimate within the samples is 



! -i^-^ 



The variance estimate between the samples is 

S M — ~ 



-^rij{xj-x) . 

s 2 estimates a 2 regardless of whether Ho is true or not. s 2 M only estimates a 2 correctly if Ho is 
true. If Hq is false, then s 2 M estimates too high. 



Now consider the statistic 



v= S -f 

S I 
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The significance probability is 



1 - ^Fisher (v) 



where Fpisher is the distribution function of Fisher's F distribution with k — 1 degrees of freedom 
in the numerator and n — k degrees of freedom in the denominator (Table B.5). 



18.3 Two examples (comparison of mean values from three samples) 

Let three samples be given: 

sample 1: 29, 28, 29, 21, 28, 22, 22, 29, 26, 26 
sample 2: 22, 21, 18, 28, 23, 25, 25, 28, 23, 26 
sample 3: 24, 23, 26, 20, 33, 23, 26, 24, 27, 22 

It is assumed that the samples originate from independent normal distributions with common 
variance. Let /ii be the expected value of the z'th normal distribution. We wish to test the null 
hypothesis 

H : iii = 1^2 = M3 • 

(As a matter of fact, all the observations originate from a normal distribution with expected value 
25 and variance 10, so the test shouldn't lead to a rejection of Hn.) We thus have k = 3 samples 
each consisting of ni = 10 observations, a total of n = 30 observations. A computation gives the 
following variance estimate within the samples: 



s] 



10.91 
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Sample number 






1 


2 




3 








29 


22 




24 








28 


21 




23 








29 


18 




26 








21 


28 




20 








28 


23 




33 








22 


25 




23 








22 


25 




26 








29 


28 




24 








26 


23 




27 








26 


26 




22 


Mean value Xj 






26.0 


23.9 




24.8 


Empirical variance s 2 - 




10.22 


9.88 




12.62 


x = 24.9 








(g 


rand mean value) 


S ] = ( s 2 + s 2 + 


^)/3 


= 10.91 




(variance within 


samples) 


S M ~ ^l^{Xj — 


x) 2 = 


11.10 




(variance between 


samples) 


v = s m/ s2 i = L02 










(statistic) 
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If we add 5 to all the observations in sample 3, we get the following table instead: 



Sample number 




1 


2 




3 






29 


22 




29 






28 


21 




28 






29 


18 




31 






21 


28 




25 






28 


23 




38 






22 


25 




28 






22 


25 




31 






29 


28 




29 






26 


23 




32 






26 


26 




27 


Mean value Xj 




26.0 


23.9 




29.8 


Empirical variance s 2 - 




10.22 


9.88 




12.62 


x = 26.6 






(grand mean value) 


s] = (si + s 2 2 + a§)/3 


= 10.91 




(variance 


within 


samples) 


S M ~ ^ Z-A^j — X) — 


89.43 




(variance between 


samples) 


v = s 2 M /sj = 8.20 










(statistic) 



Note how the variance within the samples doesn't change, whereas the variance between the 
samples is now far too large. Thus, the statistic v = 8.20 also becomes large and the significance 
probability is seen in Table B.5 to be less than 1%. Therefore, we reject the null hypothesis Ho of 
equal expected values (which was also to be expected, since Ho is now manifestly false). 

19 The chi-squared test (or x 2 test) 

19.1 x 2 test for equality of distribution 

The reason why the x 2 distribution is so important is that it can be used to test whether a given 
set of observations comes from a certain distribution. In the following sections, we shall see many 
examples of this. The test, which is also called Pearson's x 2 test or x 2 test for goodness of fit, is 
carried out as follows: 

1. First, divide the observations into categories. Let us denote the number of categories by k and 
the number of observations in the i'th category by Oi. The total number of observations is thus 
n = Ox + • • • + O k . 

2. Formulate a null hypothesis Ho. This null hypothesis must imply what the probability pi is that 
an observation belongs to the z'th category. 
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3. Compute the statistic 



X 



£ 



(Pi ~ E t 
Ei 



As mentioned, Oi is the observed number in the z'th category. Further, Ei is the expected number 
in the z'th category (expected according to the null hypothesis, that is): Ei = npi. Incidentally, 
the statistic % = \/x^ * s sometimes called the discrepancy. 



P = l- F( X 2 



4. Find the significance probability 



where F = F x 2 is the distribution function of the x 2 distribution with df degrees of freedom 
(look up in Table B.3). Hq is rejected if P is smaller than 5% (or whatever significance level one 
chooses). The number of degrees of freedom is normally df = k — 1, i.e. one less than the number 
of categories. If, however, one uses the observations to estimate the probability parameters pi of 
the null hypothesis, df becomes smaller. 



Remember: Each estimated parameter costs one degree of freedom. 



Note: It is logical to reject Hq if x 2 is large, because this implies that the difference between the 
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observed and the expected numbers is large. 

19.2 The assumption of normal distribution 

Since the x 2 test rests upon a normal approximation, it only applies provided there are not too few 
observations. 



Remember: The x 2 test applies if the expected number E{ is at least five in each category. If, 
however, there are more than five categories, an expected number of at least three in each category 
suffices. 



19.3 Standardized residuals 

If the null hypothesis regarding equality of distribution is rejected by a x 2 test, this was because 
some of the observed numbers deviated widely from the expected numbers. It is then interesting 
to investigate exactly which observed numbers are extreme. For this purpose, we compute the 

standardized residuals 

Oi-npi Oi-Ei 



y/npi{l -pi) y/Ei(l -pi) 

for each category. If the null hypothesis were true, each vi would be normally distributed with 
expected value jjl = and standard deviation a = 1. Therefore: 



Remember: Standardized residuals numerically greater than 2 are signs of an extreme observed 
number. 



It can very well happen that standardized residuals numerically greater than 2 occur even though 
the x 2 test does not lead to rejection of the null hypothesis. This does not mean that the null hy- 
pothesis should be rejected after all. In particular when one has a large number of categories, it 
will not be unusual to find some large residuals. 



Warning: Only compute the standardized residuals if the null hypothesis has been rejected by a 
X 2 test. 



19.4 Example (women with five children) 

Exercise. A hospital has registered the sex of the children of 1045 women who each have five 

children. Result: 

O r 



5 girls 58 

4 girls + 1 boy 149 
3 girls + 2 boys 305 
2 girls + 3 boys 303 
1 girl + 4 boys 162 

5 boys 45 
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Test the hypothesis Hq that, at every birth, the probability of a boy is the same as the probability 
of a girl. 

SOLUTION. If Hq is true, the above table consists of 1045 observations from a Bin(5, 1/2) distri- 
bution. The point probabilities in a Bin(5, 1/2) distribution are 

Pi 



5 girls 


0.0313 


4 girls + 1 boy 


0.1563 


3 girls + 2 boys 


0.3125 


2 girls + 3 boys 


0.3125 


1 girl + 4 boys 


0.1563 


5 boys 


0.0313 



The expected numbers E{ = 1045 • pi then become 



Et 



5 girls 


32.7 


4 girls + 1 boy 


163.3 


3 girls + 2 boys 


326.6 


2 girls + 3 boys 


326.6 


1 girl + 4 boys 


163.3 


5 boys 


32.7 



The statistic is computed: 



2 _ (58 - 32.7) 2 (149 - 163.3) 2 (305 - 326.6) 2 



32.7 163.3 326.6 

(303 - 326.6) 2 (162 - 163.3) 2 (45 - 32.7) 2 
326.6 + 163.3 + 32/7 

Since the observations are divided into six categories, we compare the statistic with the x 2 distri- 
bution with df = 6 — 1 = 5 degrees of freedom. Table B.3 shows that the significance probability 
is well below 0.5%. We can therefore with great confidence reject the hypothesis that the boy-girl 
ratio is Bin(5, 1/2) distributed. 

Let us finally compute the standardized residuals: 

n 



5 girls 


4.5 


4 girls + 1 boy 


-1.2 


3 girls + 2 boys 


-1.4 


2 girls + 3 boys 


-1.6 


1 girl + 4 boys 


-0.1 


5 boys 


2.2 



We note that it is the numbers of women with five children of the same sex which are extreme and 
make the statistic large. 
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19.5 Example (election) 

Exercise. At the election for the Danish parliament in February 2005, votes were distributed 
among the parties as follows (as percentages): 



B 



O 



V others 



25.8 9.2 10.3 6.0 13.3 29.0 3.4 3.0 

In August 2008, an opinion poll was carried out in which 1000 randomly chosen persons were 
asked which party they would vote for now. The result was: 

A B C F O V others 



242 89 98 68 141 294 43 25 
Has the popularity of the different parties changed since the election? 



Solution. We test the null hypothesis Ho that the result of the opinion poll is an observation 
from a multinomial distribution with k = 8 categories and probability parameters pi as given in 
the table above. The expected observations (given the null hypothesis) are: 



B 



O 



V others 



258 92 103 60 133 290 34 



30 
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Now we compute the statistic x 2 ' 

2 _ ^ (Pi ~ Ei? _ (242 - 258) 2 (25 - 30) 2 _ 

X "^ ^ " 258 + '" + 30 " 6 - 15 - 

By looking up in Table B.3 under the x 2 distribution with df = 8 — 1 = 7 degrees of freedom, it 
is only seen that the significance probability is below 50%. Thus, we have no statistical evidence 
to conclude that the popularity of the parties has changed. 

Let us ignore the warning in section 19.3 and compute the standardized residuals. For category 

A, for example, we find 

242-1000-0.258 
r = = = —1.16 . 

V 1000 • 0.258 • 0.742 
Altogether we get 

A B C F O V others 



-1.16 -0.33 -0.52 1.06 0.74 0.28 1.57 -0.93 
Not surprisingly, all standardized residuals are numerically smaller than 2. 

19.6 Example (deaths in the Prussian cavalry) 

In the period 1875-1894 the number of deaths caused by horse kicks was registered in 10 of the 
regiments of the Prussian cavalry. Of the total of 200 "regiment-years", there were 109 years with 
no deaths, 65 years with one death, 22 years with two deaths, three years with three deaths, and 
one year with four deaths. We wish to investigate whether these numbers come from a Poisson 
distribution Pois(A). 

In order to get expected numbers greater than five (or at least to come close to that), we group 
the years with three and four deaths into a single category and thus obtain the following observed 
numbers Oi of years with i deaths: 

i O t 






109 


1 


65 


2 


22 


>3 


4 



The intensity A is estimated as A = 122/200 = 0.61, since there were a total of 122 deaths during 
the 200 regiment-years. The point probabilities of a Pois(0.61) distribution are 

i Pi 



0.543 

1 0.331 

2 0.101 
> 3 0.024 
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The expected numbers thus become 



Ei 






108.7 


1 


66.3 


2 


20.2 


>3 


4.8 



The reader should let himself be impressed by the striking correspondence between expected and 
observed numbers. It is evidently superfluous to carry the analysis any further, but let us compute 
the statistic anyway: 

2 _ (109 - 108.7) 2 (65 - 66.3) 2 (22 - 20.2) 2 (4 - 4.8) 2 



108.7 



66.3 



20.2 



4.8 



Since there are four categories and we have estimated one parameter using the data, the statistic 
should be compared with the x 2 distribution with df = 4 — 1 — 1 = 2 degrees of freedom. As 
expected, Table B.3 shows a significance probability well above 50%. 

Incidentally, the example comes from Ladislaus von Bortkiewicz's 1898 book Das Gesetz der 
kleinen Zahlen. 
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20 Contingency tables 

20.1 Definition, method 

Suppose that a number of observations are given and that the observations are divided into cate- 
gories according to two different criteria. The number of observations in each category can then 
be displayed in a contingency table. The purpose of the test presented here is to test whether there 
is independence between the two criteria used to categorize the observations. 

Method. Let there be given an r x s table, i.e. a table with r rows and s columns: 



an 


a\2 






0>l8 


021 


«22 






&2s 






















a r \ 


a r 2 






Q rs 



It has row sums Ri = Y^ S j=i a ij> column sums Sj = Ya=i a ij> anc ^ tota l sum 
These are the observed numbers O. The row probabilities are estimated as 

Ri 



Pi - ~ N ' 



and the column probabilities as 



Pj 



N 



If there is independence between rows and columns, the cell probabilities can be estimated as 

HiOj 



Pij = Pi-P-j = 
We can thus compute the expected numbers E: 



N 2 ' 



RiSi 

N 


RiS 2 

N 






RiS s 

N 


R2S! 

N 


R2S2 

N 






R 2 S S 

N 






















R r S! 

N 


RrS 2 

N 






rC r o s 

N 
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since the expected number in the (z, j)'th cell is 

E = Np l3 = R z Sj/N . 
Now we compute the statistic 



X 



£ 



£ 



E 



RiSj/N 



where the summation is carried out over each cell of the table. If the independence hypothesis 
holds true and the expected number is at least 5 in each cell, then the statistic is x 2 distributed 
with 

d/ = (r-l)(s-l) 

degrees of freedom. 



Important! If the data are given as percentages, they must be expressed as absolute numbers 
before insertion into the contingency table. 



20.2 Standardized residuals 

If the independence hypothesis is rejected by a y 2 test, one might, as in section 19.3, be interest- 
ed in determining which cells contain observed numbers deviating extremely from the expected 
numbers. The standardized residuals are computed as 

_ Oij - RiSj/n 



(RiSj/r^il-Ri/r^Xl-Sj/n) 

If the independence hypothesis were true, each r^ would be normally distributed with expected 
value /i = and standard deviation a = 1. Standardized residuals numerically greater than 2 are 
therefore signs of an extreme observed number. 

20.3 Example (students' political orientation) 

Exercise. At three Danish universities, 488 students were asked about their faculty and which 
party they would vote for if there were to be an election tomorrow. The result (in simplified form) 
was: 





A 


B 


c 


F 





V 





Ri 


Humanities 


37 


48 


15 


26 


4 


17 


10 


157 


Natural Sci. 


32 


38 


19 


18 


7 


51 


2 


167 


Social Sci. 


32 


24 


15 


7 


12 


69 


5 


164 


Sj 


101 


110 


49 


51 


23 


137 


17 


488 



Investigate whether there is independence between the students' political orientation and their 
faculty. 
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Solution. We are dealing with a 3 x 7 table and perform a x 2 test for independence. First, the 
expected numbers 

E 



liiOj 

488 



are computed and presented in a table: 





A 


B 


C 


F 


O 


V 





Humanities 


32.5 


35.4 


15.8 


16.4 


7.4 


44.1 


5.5 


Natural Sci. 


34.6 


37.6 


16.8 


17.5 


7.9 


46.9 


5.8 


Social Sci. 


33.9 


37.0 


16.5 


17.1 


7.7 


46.0 


5.7 



Now the statistic 



X 



£ 



(O - Ef 



can be computed, since the observed numbers O are the numbers in the first table: 

^ (5 - 5.7) 2 



2 (37 -32.5V 

X =- — ^ - + 



32.5 



5.7 



60.9. 



The statistic is to be compared with a x 2 distribution with df = (3 — 1)(7 — 1) = 12 degrees of 
freedom. Table B.3 shows that the significance probability is well below 0.1%, and we therefore 
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confidently reject the independence hypothesis. 

Let us now compute the standardized residuals to see in which cells the observed numbers are 
extreme. We use the formula for r^ in section 20.2 and get 





A 


B 


C 


F 


O 


V 





Humanities 


1.1 


2.9 


-0.2 


3.0 


-1.6 


-5.8 


2.4 


Natural Sci. 


-0.6 


0.1 


0.7 


0.2 


-0.4 


0.9 


-2.0 


Social Sci. 


-0.5 


-3.0 


-0.5 


-3.2 


1.9 


4.9 


-0.4 



We find that there are extreme observations in many cells. 

20.4 x 2 test for 2 x 2 tables 

A contingency table with two rows and two columns is called a 2x2 table. Let us write the 
observed numbers as follows: 



The statistic thus becomes 



x 2 = 



ad — be 

N 



d 



1 { 1 1 } 1 

En E12 E21 E22 



where N = a + b + c + d is the total number of observations, and E^ is the expected number in the 
i j'th cell. The statistic x 2 is to be compared with the x 2 distribution with df = (2 — 1) (2 — 1) = 1 
degree of freedom. 

If we wish to perform a one-sided test of the independence hypothesis, the statistic 



u 



ad — be 

N 




1 { 1 1 | 1 

En Eu E21 E22 



is used instead. Under the independence hypothesis, u will be standard normally distributed. 

20.5 Fisher's exact test for 2 x 2 tables 

Given a 2 x 2 table, nothing stands in the way of using the x 2 test, but there is a better test in this 
situation called Fisher's exact test. Fisher's exact test does not use any normal approximation, 
and may therefore still be applied when the number of expected observations in one or more of 
the cells is smaller than five. 

Method. Let there be given a 2x2 table: 



d 

with row sums Ri = a + b and R2 = c + d and column sums Si = a + c and S2 = b + d and total 
sum N = Ri + R2 = Si + S2 = a + b + c + d. We test the independence hypothesis Hq against 
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the alternative hypothesis Hi that the "diagonal probabilities" pu and P22 are greater than what 
they would have been had there been independence. (This situation can always be arranged by 
switching the rows if necessary.) The conditional probability of obtaining exactly the 2x2 table 
above, given that the row sums are R\ and R 2 , and that the column sums are S\ and S2, is 

_ R 1 \R 2 \S 1 \S 2 \ 

^conditional" jy, fl , ft , d d j ■ 

The significance probability in Fisher's exact test is the sum of ^conditional taken on all 2x2 tables 
with the same row and column sums as in the given table, and which are at least as extreme as the 
given table: 

p _ mi S' c} R 1 \R 2 \S 1 \S 2 \ 

Fisher ^ N\ (a + i)\ (b-i)l(c-i)l(d + i)l' 

The independence hypothesis Hq is rejected if PFisher is smaller than 5% (or whatever signifi- 
cance level one has chosen). 

Addendum: If a two-sided test is performed, i.e. if one does not test against any specific alter- 
native hypothesis, the significance probability becomes 2 • PFisher- It is then necessary that the 
2x2 table is written in such a way that the observed numbers in the diagonal are greater than the 
expected numbers (this can always be obtained by switching the rows if necessary). 

20.6 Example (Fisher's exact test) 

In a medical experiment concerning alternative treatments, ten patients are randomly divided into 
two groups with five patients in each. The patients in the first group receive acupuncture, while 
the patients in the other group receive no treatment. It is then seen which patients are fit or ill at 
the end of the experiment. The result can be presented in a 2 x 2 table: 

fit ill 



acupuncture 
no treatment 



4 1 

2 3 



The significance probability in Fisher's exact test is computed as 

p _V 5!5!6!4! _ofi« 

Fisher ~ ^ 10! (4 + i)\ (1 - i)\ (2 - i)\ (3 + i)\ ~ /0 ' 

With such a large significance probability, there is no evidence that acupuncture had any effect. 

21 Distribution-free tests 

In all tests considered so far, we have known something about the distribution from which the 
given samples originated. We knew, for example, that the distribution was a normal distribution 
even though we didn't know the expected value or the standard deviation. 

Sometimes, though, one knows nothing at all about the underlying distribution. It then be- 
comes necessary to use a distribution-free test (also known as a non-parametric test). The two 
examples considered in this chapter are due to Frank Wilcoxon (1892-1965). 



Download free eBooks at bookboon.com 

74 



Statistics 



Distribution-free costs 



21.1 Wilcoxon's test for one set of observations 

Let there be given n independent observations d\ , . . . , d n from an unknown distribution. We test 
the null hypothesis 

Hq: The unknown distribution is symmetric around 0. 

Each observation di is given a rank which is one of the numbers 1, 2, . . . , n. The observation 
with the smallest numerical value is assigned rank 1, the observation with the second smallest 
numerical value is assigned rank 2, etc. Now define the statistics 

£+ = ^ (ranks corresponding to positive di), 
t- = ^ (ranks corresponding to negative di). 

(One can check at this point whether £+ + £_ = n(n + l)/2; if not, one has added the numbers 
incorrectly.) If Hq holds true, then t + and £_ should be more or less equal. When to reject Hq 
depends on which alternative hypothesis is tested against. 

If we test Hq against the alternative hypothesis 

Hi: The unknown distribution primarily gives positive observations, 

then Hq is rejected if £_ is extremely small. Choose a significance level a and consult Table B.8 
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under n and a. If £_ is smaller than or equal to the table value, Hq is rejected. If £_ is greater than 
the table value, Ho is accepted. 

If we test Ho against the alternative hypothesis 

Hi: The unknown distribution primarily gives negative observations, 

then Ho is rejected if t+ is extremely small. Choose a significance level a and consult Table B.8 
under n and a. If t + is smaller than or equal to the table value, Hq is rejected. If t + is greater than 
the table value, Hq is accepted. 

If we don't test Hq against any particular alternative hypothesis, the null hypothesis is rejected 
if the minimum t \— min{£ + , £_} is extremely small. Choose a significance level a and consult 
Table B.8 under n and a/2 (if, for example, we choose the significance level a = 5%, then we 
look up in the table under n and 0.025). If t is smaller than or equal to the table value, we reject 
Hq. If t is greater than the table value, we accept Hq. 

The above test applies in particular when two sets of observations xi, . . . , x n and yi, . . . , y n 
are given and di is the difference between the "before values" Xi and the "after values" yi, i.e. 
di = Xi — yi. If there are only random, unsystematic differences between the before and after 
values, it follows that the g^'s are distributed symmetrically around 0. 

21.2 Example 

An experiment involving ten patients is carried out to determine whether physical exercise lowers 
blood pressure. At the beginning of the experiment, the patients' blood pressures are measured. 
These observations are denoted xi, . . . , xio. After a month of exercise, the blood pressures are 
measured again. These observations are denoted yi, . . . , yio- We now test the null hypothesis 

Hq: Physical exercise has no influence on blood pressure. The ten differences d{ — X{ — yi are 
therefore distributed symmetrically around 0, 

against the alternative hypothesis 

Hi: Physical exercise causes the blood pressure to decrease. The ten differences di are therefore 
primarily positive. 

We compute the ranks and t+ and £_: 

Person 123456789 10 

Before x { 140 125 110 130 170 165 135 140 155 145~ 

After yi 137 137 102 104 172 125 140 110 140 126 

Difference rfi 3 -12 8 26 -2 40 -5 30 15 19 

Rank 25481 10 3967 

t + = 2 + 4 + 6 + 7 + 8 + 9 + 10 = 46, 

t_ = 1 + 3 + 5 = 9. 
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We shall reject Ho if t- = 9 is extremely small. Table B.8 with significance level a = 5% shows 
that "extremely small" means ^10. Conclusion: The test shows that the null hypothesis Hq must 
be rejected against the alternative hypothesis Hi at significance level 5%. 

21.3 The normal approximation to Wilcoxon's test for one set of observations 

Table B.8 includes values up to n = 50. If the number of observations is greater, a normal dis- 
tribution approximation can be applied. Indeed, if the null hypothesis is true, the statistic £+ is 
approximately normally distributed with expected value 

_ n(n+ 1) 



and standard deviation 



ln(n + l)(2n + l) 



24 
The significance probability is therefore found by comparison of the statistic 

a 
with Table B.2 of the standard normal distribution. 

Example. Let us use the normal approximation to find the significance probability in the previous 
example (even though n here is smaller than 50 and the approximation therefore is not highly 
precise). We get /i = 27.5 and a = 9.81. The statistic therefore becomes z = 1.89, which gives 
a significance probability of 2.9%. The conclusion is thus the same, namely that Hq is rejected at 
significance level 5%. 

21.4 Wilcoxon's test for two sets of observations 

Suppose we have two sets xi, . . . , x n and yi, . . . , y m of independent observations. We test the 
null hypothesis 

Hq: The observations come from the same distribution. 

Each of the n + m observations is assigned a rank which is one of the numbers 1, 2, . . . , n + m. 
The observation with the smallest numerical value is assigned rank 1, the observation with the 
second smallest numerical value is assigned rank 2, etc. Define the statistic 

t x = XX ran ks of the Xi's). 

Whether Hq is rejected or not depends on which alternative hypothesis we test against. 

If we test Hq against the alternative hypothesis 

Hi : The X{ 's are primarily smaller than the yi 's, 

then Hq is rejected if t x is extremely small. Look up in Table B.9 under n and m. If t x is smaller 
than or equal to the table value, then Hq is rejected at significance level a = 5%. If t x is greater 
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than the table value, then Hq is accepted at significance level a = 5%. 

If we test Hq against the alternative hypothesis 

Hi : The x\ 's are primarily greater than the yi 's, 

then one has to switch the roles of o^'s and y^'s and continue as described above. 

If we don't test Hq against any particular alternative hypothesis, then the null hypothesis is reject- 
ed if the minimum 

t := mm{t x , n{n + m + 1) — t x } 

is extremely small. Look up in Table B.9 under n and m. If t is smaller than or equal to the table 
value, then Hq is rejected at significance level a = 10%. If t is greater than the table value, then 
Hq is accepted at significance level 10%. 

21.5 The normal approximation to Wilcoxon's test for two sets of observations 

Table B.9 applies for moderate values of n and m. If the number of observations is greater, one 
can use a normal distribution approximation. Indeed, if the null hypothesis holds true, the statistic 
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t x is approximately normally distributed with expected value 

n(n + 771 + 1) 



and standard deviation 



fi 



a = 



nm(n + m + 1) 
12 



The significance probability is then found by comparing the statistic 

t x - ti 



a 



with Table B.2 of the standard normal distribution. 



22 Linear regression 

22.1 The model 

Suppose we have a sample consisting of n pairs of observations 

Ol,yi), (x 2 ,y 2 ),..., (XrnVn) • 

We propose the model that each yi is an observation from a random variable 

Yi = /?o + to + #; 

where the i^'s are independent normally distributed random variables with expected value and 
common variance a 2 . Thus we can express each yi as 

yi=Po + foxi + ei 

where e 2 - is an observation from E{. We call yi the response variable, X{ the declaring variable 
and e{ the remainder term. 

22.2 Estimation of the parameters /3q and fa 

Let x be the mean value of the x^'s and y the mean value of the y z 's. Define the sum of products 
of errors as 



2=1 



SPE 1 ^ = ^0; - x)(j/i - y) 
and the sum of squares of errors as 

SSE X = J2(* 



(Xi - x 



2=1 



The parameters /?o and f3\ of the regression equation are now estimated as 



SPE 



xy 



SSE X 
Po = y - fax 
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22.3 The distribution of the estimators 

If the model's assumptions are met, the estimator /3o is normally distributed with expected value 
Po (the estimator thus is unbiased) and variance a 2 (l/n + x 2 /SSE x ). In other words, it holds 
that 

A,~*( A ,,»(i + 5 *L 

Moreover, the estimator $1 is normally distributed with expected value f3\ (this estimator is there- 
fore unbiased too) and variance a 2 /SSE x . In other words, it holds that 



SSE X 

22 A Predicted values yi and residuals e^ 

From the estimates /3o and /?i, the predicted value of yi can be computed for each i as 

m = A) + PiXi . 

The i'th residual e\ is the difference between the actual value y\ and the predicted value yf. 

ei = Vi-yi • 
The residual ei is an estimate of the remainder term e^. 

22.5 Estimation of the variance a 2 

We introduce the sum of squares of residuals as 

n 
i=l 

The variance a 2 of the remainder terms is now estimated as 

9 SSR 

s = ~ • 

n-2 

This estimator is unbiased (but different from the maximum likelihood estimator). 

22.6 Confidence intervals for the parameters (3 and p 1 

After estimating the parameters /% and /3i, we can compute the confidence intervals with confi- 
dence level 1 — a around the estimates $0 and (3\. These are 



* s 

h±tl - al2 7MW x 

The number t 1 _ a / 2 is determined by F(ui_ a / 2 ) = 1 — ce/2, where F is the distribution function 
of Student's t distribution with n — 1 degrees of freedom (see also section 15.8). 
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22.7 The determination coefficient R 2 

In order to investigate how well the model with the estimated parameters describes the actual 
observations, we compute the determination coefficient 



R 



2 SSEy — SSR 
SSEa, 



R 2 lies in the interval [0, 1] and measures the part of the variation of the y^'s which the model 
describes as a linear function of the x ? 's. 



Remember: The greater the determination coefficient R 2 is, the better the model describes the 
observations. 



22.8 Predictions and prediction intervals 

Let there be given a real number xq. The function value 

yo = A) + Pixo 

is then estimated, or predicted, as 

yo = A) + Pixq . 
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The confidence interval, or prediction interval, with confidence level 1 — a around the estimate 

do is 

/ 1 (%o — x) 2 

The number t 1 _ a / 2 is determined by F(u 1 _ a / 2 ) = 1 — a/2, where F is the distribution function 
of Student's t distribution with n — 2 degrees of freedom (see also section 15.8). 

22.9 Overview of formulae 

S x = Ya=i Xi The sum °^ ^ e x *' s 

x = S x /n The mean value of the a^'s 

SS X = Ya=i x 1 The sum °^ ^ e squares of the o^'s 

SSE X = X^=i( x * ~~ ^) 2 = SS X — Sl/n The sum of the squares of the errors 

s 2 = SSE x /(n — 1) Empirical variance of the o^'s 

SP xy = Y^i=i x iVi The sum °f th e products 

SPE xy = X^=i( x * ~~ ^)(y* ~~ y) = S^y ~~ S x S y /n The sum of the products of the errors 

ft = SPE xy /SSE x The estimate of ft 

ft = y — ft x The estimate of ft 

j/i = 6eo + ft x^ Predicted value of ^ 

&i = Vi — Vi The z'th residual 

SSi? = Y% =1 e? = SS^ - SPEl y /SSE x The sum of the squares of the residuals 

5 2 = SSR/(n - 2) The estimate a 2 

i? 2 = 1 - SSR/SSEy The determination coefficient 

22.10 Example 

Exercise. It is claimed that the temperature in the Andes Mountains decreases by six degrees per 
1000 metres. The following temperatures were measured simultaneously at ten different localities 
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in the same region: 



Altitude Xi 


Temperature yi 


(metres) 


(degrees) 


500 


15 


1000 


14 


1500 


11 


2000 


6 


2500 


-1 


3000 


2 


3500 





4000 


-4 


4500 


-8 


5000 


-14 



We use a linear regression model 



y% = A) + PiXi + ei 



where the remainder terms ei are independent normally distributed with expected value and the 
same (unknown) variance a 2 . 

1) Estimate the parameters (3q and f3\. 

2) Determine the confidence interval with confidence level 95% for f3\. 

3) Can the hypothesis Ho : (3\ = —0.006 be accepted? 

4) To how large degree can the difference of temperature be explained as a linear function of the 
altitude? 



Solution. First we perform the relevant computations: 

S x = E]=i*i = 27500 

x = S x /10 = 2750 

SS X = Y,]=i A = 96250000 

SSE X = SS X - Si/ 10 = 20625000 

SPxy = El=i *iVi = "68500 



/3 X = SPE xy /SSE x = -0.0061 



42.1 



Sy = 




= 21 


y = 


Sy/W = 2.1 


SSy 


-V 10 v 2 
— 2-,i=\ Ui 


= 859 


D L) -tL/y = O Uy 


52/10 = 81 


D 1 Jl/ X y = U l X y 


- 5^5^/10 


Po = 


= y - fax = 


18.9 


s 2 = 


-- SSR/8 = 


5.26 



-126250 



SSR — SSEy — SPE xy / SSE X 
E 2 = 1 - SSR/ SSEy = 0.948 

1) It appears directly from the computations that the estimates of /3q and f3\ are 

$0 = 18.9 , 1 = -0.0061 . 
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2) Table B.4, under df = 10 — 1 = 9 degrees of freedom, shows that to. 975 = 2.26 (see also 
section 15.8). The confidence interval around (3\ thus becomes 



-0.0061-2.26^2= , -0.0061 + 2.26- V ^ 



V20625000 



V20625000 



[-0.0072 , -0.0050] . 



3) The hypothesis Ho : /?i = —0.006 is accepted, since this value lies within the confidence 
interval. 

4) The part of the temperature difference describable as a linear function of the altitude is nothing 
other than the determination coefficient 

R 2 = 94.8% . 

The fact that R 2 is large (close to 100%) shows that the actual temperatures are quite close to those 
predicted. This also appears from the figure below, which shows that the actual temperatures are 
very close to the regression line: 
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A Overview of discrete distributions 



Mean 
value 



Distribution 



Description 



Values 



Point proba- 
bilities 



Variance 



Binomial 
distribution 

Bin(n,p) 



Number of 
successes in 
n tries 



k = 0, 1, . . . ,n 



n 
k 



k „n—k 



jrq 



np 



npq 



Poisson 
distribution 

Pois(A) 



Number of 
spontaneous 
events in a 
time interval 



fc = 0,l,. 



A* -A 

k\ e 



Geometrical 
distribution 

Geo(p) 



Number 
of fail- 
ures before 
success 



fc = 0,l,. 



q k p 



q/p 



q/p 



Hyper- 
geometrical 
distribution 

HG(n, r, N) 



Number of 
red balls 
among n 
balls 



k = 0,... 
. . . , min{n, r} 



r s 

kj \n — k 

n 



nr/N 



nrs(N-n) 
N' 2 (N-1) 



Negative 
binomial 
distribution 

NB(n,p) 



Number 
of failures 
before the 
n'th success 



fc = 0,l,... 



n + k-1 \ k 

• p r • q* 
n — 1 / 



nq/p 



nq/p 2 



Multi- 
nomial- 
distribution 
Mult(n, . . . ) 



Number 
of sample 
points of 
each type 



(/ci, . . . , k r ) 
where 



n 



*.-*, nrf ' 
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B Tables 

B.l How to read the tables 

Table B.2 gives values of the distribution function 

* w= /-L^ exp ( 

of the standard normal distribution. 



--/ 



dt 



Table B.3 gives values of x for which the distribution function F = F x 2 of the x 2 distribution 
with df degrees of freedom takes the values F(x) = 0.500, F(x) = 0.600, etc. 

Table B.4 gives values of x for which the distribution function F = Fstudent of Student's t distri- 
bution with df degrees of freedom takes the values F(x) = 0.600, F(x) = 0.700, etc. 

Table B.5, Table B.6 and Table B.7 give values of x for which the distribution function F = 
^Fisher °f Fisher's F distribution with n degrees of freedom in the numerator (top line) and 
m degrees of freedom in the denominator (leftmost column) takes the values F{x) = 0.90, 
F(x) = 0.95, and F{x) = 0.99, respectively. 
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Table B.8 and Table B.9 give critical values for Wilcoxon's tests for one and two sets of observa- 
tions. See Chapter 21 for further details. 
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B.2 The standard normal distribution 

u *(u) <&(— u) u 3>(u) <£(— u) u 3>(u) <&(— u) 



0.00 0.5000 0.5000 0.36 0.6406 0.3594 0.72 0.7642 0.2358 

0.01 0.5040 0.4960 0.37 0.6443 0.3557 0.73 0.7673 0.2327 

0.02 0.5080 0.4920 0.38 0.6480 0.3520 0.74 0.7704 0.2296 

0.03 0.5120 0.4880 0.39 0.6517 0.3483 0.75 0.7734 0.2266 

0.04 0.5160 0.4840 0.40 0.6554 0.3446 0.76 0.7764 0.2236 

0.05 0.5199 0.4801 0.41 0.6591 0.3409 0.77 0.7794 0.2206 

0.06 0.5239 0.4761 0.42 0.6628 0.3372 0.78 0.7823 0.2177 

0.07 0.5279 0.4721 0.43 0.6664 0.3336 0.79 0.7852 0.2148 

0.08 0.5319 0.4681 0.44 0.6700 0.3300 0.80 0.7881 0.2119 

0.09 0.5359 0.4641 0.45 0.6736 0.3264 0.81 0.7910 0.2090 

0.10 0.5398 0.4602 0.46 0.6772 0.3228 0.82 0.7939 0.2061 

0.11 0.5438 0.4562 0.47 0.6808 0.3192 0.83 0.7967 0.2033 

0.12 0.5478 0.4522 0.48 0.6844 0.3156 0.84 0.7995 0.2005 

0.13 0.5517 0.4483 0.49 0.6879 0.3121 0.85 0.8023 0.1977 

0.14 0.5557 0.4443 0.50 0.6915 0.3085 0.86 0.8051 0.1949 

0.15 0.5596 0.4404 0.51 0.6950 0.3050 0.87 0.8078 0.1922 

0.16 0.5636 0.4364 0.52 0.6985 0.3015 0.88 0.8106 0.1894 

0.17 0.5675 0.4325 0.53 0.7019 0.2981 0.89 0.8133 0.1867 

0.18 0.5714 0.4286 0.54 0.7054 0.2946 0.90 0.8159 0.1841 

0.19 0.5753 0.4247 0.55 0.7088 0.2912 0.91 0.8186 0.1814 

0.20 0.5793 0.4207 0.56 0.7123 0.2877 0.92 0.8212 0.1788 

0.21 0.5832 0.4168 0.57 0.7157 0.2843 0.93 0.8238 0.1762 

0.22 0.5871 0.4129 0.58 0.7190 0.2810 0.94 0.8264 0.1736 

0.23 0.5910 0.4090 0.59 0.7224 0.2776 0.95 0.8289 0.1711 

0.24 0.5948 0.4052 0.60 0.7257 0.2743 0.96 0.8315 0.1685 

0.25 0.5987 0.4013 0.61 0.7291 0.2709 0.97 0.8340 0.1660 

0.26 0.6026 0.3974 0.62 0.7324 0.2676 0.98 0.8365 0.1635 

0.27 0.6064 0.3936 0.63 0.7357 0.2643 0.99 0.8389 0.1611 

0.28 0.6103 0.3897 0.64 0.7389 0.2611 1.00 0.8413 0.1587 

0.29 0.6141 0.3859 0.65 0.7422 0.2578 1.01 0.8438 0.1562 

0.30 0.6179 0.3821 0.66 0.7454 0.2546 1.02 0.8461 0.1539 

0.31 0.6217 0.3783 0.67 0.7486 0.2514 1.03 0.8485 0.1515 

0.32 0.6255 0.3745 0.68 0.7517 0.2483 1.04 0.8508 0.1492 

0.33 0.6293 0.3707 0.69 0.7549 0.2451 1.05 0.8531 0.1469 

0.34 0.6331 0.3669 0.70 0.7580 0.2420 1.06 0.8554 0.1446 

0.35 0.6368 0.3632 0.71 0.7611 0.2389 1.07 0.8577 0.1423 
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u *(u) <&(— u) u *(u) <&(— u) u *(u) $(— u) 



1.08 0.8599 0.1401 1.45 0.9265 0.0735 1.82 0.9656 0.0344 

1.09 0.8621 0.1379 1.46 0.9279 0.0721 1.83 0.9664 0.0336 

1.10 0.8643 0.1357 1.47 0.9292 0.0708 1.84 0.9671 0.0329 

1.11 0.8665 0.1335 1.48 0.9306 0.0694 1.85 0.9678 0.0322 

1.12 0.8686 0.1314 1.49 0.9319 0.0681 1.86 0.9686 0.0314 

1.13 0.8708 0.1292 1.50 0.9332 0.0668 1.87 0.9693 0.0307 

1.14 0.8729 0.1271 1.51 0.9345 0.0655 1.88 0.9699 0.0301 

1.15 0.8749 0.1251 1.52 0.9357 0.0643 1.89 0.9706 0.0294 

1.16 0.8770 0.1230 1.53 0.9370 0.0630 1.90 0.9713 0.0287 

1.17 0.8790 0.1210 1.54 0.9382 0.0618 1.91 0.9719 0.0281 

1.18 0.8810 0.1190 1.55 0.9394 0.0606 1.92 0.9726 0.0274 

1.19 0.8830 0.1170 1.56 0.9406 0.0594 1.93 0.9732 0.0268 

1.20 0.8849 0.1151 1.57 0.9418 0.0582 1.94 0.9738 0.0262 

1.21 0.8869 0.1131 1.58 0.9429 0.0571 1.95 0.9744 0.0256 

1.22 0.8888 0.1112 1.59 0.9441 0.0559 1.96 0.9750 0.0250 

1.23 0.8907 0.1093 1.60 0.9452 0.0548 1.97 0.9756 0.0244 

1.24 0.8925 0.1075 1.61 0.9463 0.0537 1.98 0.9761 0.0239 

1.25 0.8944 0.1056 1.62 0.9474 0.0526 1.99 0.9767 0.0233 

1.26 0.8962 0.1038 1.63 0.9484 0.0516 2.00 0.9772 0.0228 

1.27 0.8980 0.1020 1.64 0.9495 0.0505 2.01 0.9778 0.0222 

1.28 0.8997 0.1003 1.65 0.9505 0.0495 2.02 0.9783 0.0217 

1.29 0.9015 0.0985 1.66 0.9515 0.0485 2.03 0.9788 0.0212 

1.30 0.9032 0.0968 1.67 0.9525 0.0475 2.04 0.9793 0.0207 

1.31 0.9049 0.0951 1.68 0.9535 0.0465 2.05 0.9798 0.0202 

1.32 0.9066 0.0934 1.69 0.9545 0.0455 2.06 0.9803 0.0197 

1.33 0.9082 0.0918 1.70 0.9554 0.0446 2.07 0.9808 0.0192 

1.34 0.9099 0.0901 1.71 0.9564 0.0436 2.08 0.9812 0.0188 

1.35 0.9115 0.0885 1.72 0.9573 0.0427 2.09 0.9817 0.0183 

1.36 0.9131 0.0869 1.73 0.9582 0.0418 2.10 0.9821 0.0179 

1.37 0.9147 0.0853 1.74 0.9591 0.0409 2.11 0.9826 0.0174 

1.38 0.9162 0.0838 1.75 0.9599 0.0401 2.12 0.9830 0.0170 

1.39 0.9177 0.0823 1.76 0.9608 0.0392 2.13 0.9834 0.0166 

1.40 0.9192 0.0808 1.77 0.9616 0.0384 2.14 0.9838 0.0162 

1.41 0.9207 0.0793 1.78 0.9625 0.0375 2.15 0.9842 0.0158 

1.42 0.9222 0.0778 1.79 0.9633 0.0367 2.16 0.9846 0.0154 

1.43 0.9236 0.0764 1.80 0.9641 0.0359 2.17 0.9850 0.0150 

1.44 0.9251 0.0749 1.81 0.9649 0.0351 2.18 0.9854 0.0146 
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u *(u) <&(— u) u *(u) <&(— u) u *(u) $(— u) 



2.19 0.9857 0.0143 2.56 0.9948 0.0052 2.93 0.9983 0.0017 

2.20 0.9861 0.0139 2.57 0.9949 0.0051 2.94 0.9984 0.0016 

2.21 0.9864 0.0136 2.58 0.9951 0.0049 2.95 0.9984 0.0016 

2.22 0.9868 0.0132 2.59 0.9952 0.0048 2.96 0.9985 0.0015 

2.23 0.9871 0.0129 2.60 0.9953 0.0047 2.97 0.9985 0.0015 

2.24 0.9875 0.0125 2.61 0.9955 0.0045 2.98 0.9986 0.0014 

2.25 0.9878 0.0122 2.62 0.9956 0.0044 2.99 0.9986 0.0014 

2.26 0.9881 0.0119 2.63 0.9957 0.0043 3.00 0.9987 0.0013 

2.27 0.9884 0.0116 2.64 0.9959 0.0041 3.10 0.9990 0.0010 

2.28 0.9887 0.0113 2.65 0.9960 0.0040 3.20 0.9993 0.0007 

2.29 0.9890 0.0110 2.66 0.9961 0.0039 3.30 0.9995 0.0005 

2.30 0.9893 0.0107 2.67 0.9962 0.0038 3.40 0.9997 0.0003 

2.31 0.9896 0.0104 2.68 0.9963 0.0037 3.50 0.9998 0.0002 

2.32 0.9898 0.0102 2.69 0.9964 0.0036 3.60 0.9998 0.0002 

2.33 0.9901 0.0099 2.70 0.9965 0.0035 3.70 0.9999 0.0001 

2.34 0.9904 0.0096 2.71 0.9966 0.0034 3.80 0.9999 0.0001 

2.35 0.9906 0.0094 2.72 0.9967 0.0033 3.90 1.0000 0.0000 

2.36 0.9909 0.0091 2.73 0.9968 0.0032 4.00 1.0000 0.0000 

2.37 0.9911 0.0089 2.74 0.9969 0.0031 

2.38 0.9913 0.0087 2.75 0.9970 0.0030 

2.39 0.9916 0.0084 2.76 0.9971 0.0029 

2.40 0.9918 0.0082 2.77 0.9972 0.0028 

2.41 0.9920 0.0080 2.78 0.9973 0.0027 

2.42 0.9922 0.0078 2.79 0.9974 0.0026 

2.43 0.9925 0.0075 2.80 0.9974 0.0026 

2.44 0.9927 0.0073 2.81 0.9975 0.0025 

2.45 0.9929 0.0071 2.82 0.9976 0.0024 

2.46 0.9931 0.0069 2.83 0.9977 0.0023 

2.47 0.9932 0.0068 2.84 0.9977 0.0023 

2.48 0.9934 0.0066 2.85 0.9978 0.0022 

2.49 0.9936 0.0064 2.86 0.9979 0.0021 

2.50 0.9938 0.0062 2.87 0.9979 0.0021 

2.51 0.9940 0.0060 2.88 0.9980 0.0020 

2.52 0.9941 0.0059 2.89 0.9981 0.0019 

2.53 0.9943 0.0057 2.90 0.9981 0.0019 

2.54 0.9945 0.0055 2.91 0.9982 0.0018 

2.55 0.9946 0.0054 2.92 0.9982 0.0018 
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B.3 The x 2 distribution (values x with F x 2(x) = 0.500 etc.) 



df 0.500 0.600 0.700 0.800 0.900 0.950 0.975 0.990 0.995 0.999 



1 


0.45 


0.71 


1.07 


1.64 


2.71 


3.84 


5.02 


6.63 


7.88 


10.83 


2 


1.39 


1.83 


2.41 


3.22 


4.61 


5.99 


7.38 


9.21 


10.60 


13.82 


3 


2.37 


2.95 


3.66 


4.64 


6.25 


7.81 


9.35 


11.34 


12.84 


16.27 


4 


3.36 


4.04 


4.88 


5.99 


7.78 


9.49 


11.14 


13.28 


14.86 


18.47 


5 


4.35 


5.13 


6.06 


7.29 


9.24 


11.07 


12.83 


15.09 


16.75 


20.52 


6 


5.35 


6.21 


7.23 


8.56 


10.64 


12.59 


14.45 


16.81 


18.55 


22.46 


7 


6.35 


7.28 


8.38 


9.80 


12.02 


14.07 


16.01 


18.48 


20.28 


24.32 


8 


7.34 


8.35 


9.52 


11.03 


13.36 


15.51 


17.53 


20.09 


21.95 


26.12 


9 


8.34 


9.41 


10.66 


12.24 


14.68 


16.92 


19.02 


21.67 


23.59 


27.88 


10 


9.34 


10.47 


11.78 


13.44 


15.99 


18.31 


20.48 


23.21 


25.19 


29.59 


11 


10.34 


11.53 


12.90 


14.63 


17.28 


19.68 


21.92 


24.72 


26.76 


31.26 


12 


11.34 


12.58 


14.01 


15.81 


18.55 


21.03 


23.34 


26.22 


28.30 


32.91 


13 


12.34 


13.64 


15.12 


16.98 


19.81 


22.36 


24.74 


27.69 


29.82 


34.53 


14 


13.34 


14.69 


16.22 


18.15 


21.06 


23.68 


26.12 


29.14 


31.32 


36.12 


15 


14.34 


15.73 


17.32 


19.31 


22.31 


25.00 


27.49 


30.58 


32.80 


37.70 


16 


15.34 


16.78 


18.42 


20.47 


23.54 


26.30 


28.85 


32.00 


34.27 


39.25 


17 


16.34 


17.82 


19.51 


21.61 


24.77 


27.59 


30.19 


33.41 


35.72 


40.79 


18 


17.34 


18.87 


20.60 


22.76 


25.99 


28.87 


31.53 


34.81 


37.16 


42.31 


19 


18.34 


19.91 


21.69 


23.90 


27.20 


30.14 


32.85 


36.19 


38.58 


43.82 


20 


19.34 


20.95 


22.77 


25.04 


28.41 


31.41 


34.17 


37.57 


40.00 


45.31 


21 


20.34 


21.99 


23.86 


26.17 


29.62 


32.67 


35.48 


38.93 


41.40 


46.80 


22 


21.34 


23.03 


24.94 


27.30 


30.81 


33.92 


36.78 


40.29 


42.80 


48.27 


23 


22.34 


24.07 


26.02 


28.43 


32.01 


35.17 


38.08 


41.64 


44.18 


49.73 


24 


23.34 


25.11 


27.10 


29.55 


33.20 


36.42 


39.36 


42.98 


45.56 


51.18 


25 


24.34 


26.14 


28.17 


30.68 


34.38 


37.65 


40.65 


44.31 


46.93 


52.62 


26 


25.34 


27.18 


29.25 


31.79 


35.56 


38.89 


41.92 


45.64 


48.29 


54.05 


27 


26.34 


28.21 


30.32 


32.91 


36.74 


40.11 


43.19 


46.96 


49.64 


55.48 


28 


27.34 


29.25 


31.39 


34.03 


37.92 


41.34 


44.46 


48.28 


50.99 


56.89 


29 


28.34 


30.28 


32.46 


35.14 


39.09 


42.56 


45.72 


49.59 


52.34 


58.30 


30 


29.34 


31.32 


33.53 


36.25 


40.26 


43.77 


46.98 


50.89 


53.67 


59.70 


31 


30.34 


32.35 


34.60 


37.36 


41.42 


44.99 


48.23 


52.19 


55.00 


61.10 


32 


31.34 


33.38 


35.66 


38.47 


42.58 


46.19 


49.48 


53.49 


56.33 


62.49 


33 


32.34 


34.41 


36.73 


39.57 


43.75 


47.40 


50.73 


54.78 


57.65 


63.87 


34 


33.34 


35.44 


37.80 


40.68 


44.90 


48.60 


51.97 


56.06 


58.96 


65.25 


35 


34.34 


36.47 


38.86 


41.78 


46.06 


49.80 


53.20 


57.34 


60.27 


66.62 
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df 


0.500 


0.600 


0.700 


0.800 


0.900 


0.950 


0.975 


0.990 


0.995 


0.999 


36 


35.34 


37.50 


39.92 


42.88 


47.21 


51.00 


54.44 


58.62 


61.58 


67.99 


37 


36.34 


38.53 


40.98 


43.98 


48.36 


52.19 


55.67 


59.89 


62.88 


69.35 


38 


37.34 


39.56 


42.05 


45.08 


49.51 


53.38 


56.90 


61.16 


64.18 


70.70 


39 


38.34 


40.59 


43.11 


46.17 


50.66 


54.57 


58.12 


62.43 


65.48 


72.05 


40 


39.34 


41.62 


44.16 


47.27 


51.81 


55.76 


59.34 


63.69 


66.77 


73.40 


41 


40.34 


42.65 


45.22 


48.36 


52.95 


56.94 


60.56 


64.95 


68.05 


74.74 


42 


41.34 


43.68 


46.28 


49.46 


54.09 


58.12 


61.78 


66.21 


69.34 


76.08 


43 


42.34 


44.71 


47.34 


50.55 


55.23 


59.30 


62.99 


67.46 


70.62 


77.42 


44 


43.34 


45.73 


48.40 


51.64 


56.37 


60.48 


64.20 


68.71 


71.89 


78.75 


45 


44.34 


46.76 


49.45 


52.73 


57.51 


61.66 


65.41 


69.96 


73.17 


80.08 


46 


45.34 


47.79 


50.51 


53.82 


58.64 


62.83 


66.62 


71.20 


74.44 


81.40 


47 


46.34 


48.81 


51.56 


54.91 


59.77 


64.00 


67.82 


72.44 


75.70 


82.72 


48 


47.34 


49.84 


52.62 


55.99 


60.91 


65.17 


69.02 


73.68 


76.97 


84.04 


49 


48.33 


50.87 


53.67 


57.08 


62.04 


66.34 


70.22 


74.92 


78.23 


85.35 


50 


49.33 


51.89 


54.72 


58.16 


63.17 


67.50 


71.42 


76.15 


79.49 


86.66 


51 


50.33 


52.92 


55.78 


59.25 


64.30 


68.67 


72.62 


77.39 


80.75 


87.97 


52 


51.33 


53.94 


56.83 


60.33 


65.42 


69.83 


73.81 


78.62 


82.00 


89.27 


53 


52.33 


54.97 


57.88 


61.41 


66.55 


70.99 


75.00 


79.84 


83.25 


90.57 


54 


53.33 


55.99 


58.93 


62.50 


67.67 


72.15 


76.19 


81.07 


84.50 


91.87 


55 


54.33 


57.02 


59.98 


63.58 


68.80 


73.31 


77.38 


82.29 


85.75 


93.17 


56 


55.33 


58.04 


61.03 


64.66 


69.92 


74.47 


78.57 


83.51 


86.99 


94.46 


57 


56.33 


59.06 


62.08 


65.74 


71.04 


75.62 


79.75 


84.73 


88.24 


95.75 


58 


57.33 


60.09 


63.13 


66.82 


72.16 


76.78 


80.94 


85.95 


89.48 


97.04 


59 


58.33 


61.11 


64.18 


67.89 


73.28 


77.93 


82.12 


87.17 


90.72 


98.32 


60 


59.33 


62.13 


65.23 


68.97 


74.40 


79.08 


83.30 


88.38 


91.95 


99.61 


61 


60.33 


63.16 


66.27 


70.05 


75.51 


80.23 


84.48 


89.59 


93.19 


100.89 


62 


61.33 


64.18 


67.32 


71.13 


76.63 


81.38 


85.65 


90.80 


94.42 


102.17 


63 


62.33 


65.20 


68.37 


72.20 


77.75 


82.53 


86.83 


92.01 


95.65 


103.44 


64 


63.33 


66.23 


69.42 


73.28 


78.86 


83.68 


88.00 


93.22 


96.88 


104.72 


65 


64.33 


67.25 


70.46 


74.35 


79.97 


84.82 


89.18 


94.42 


98.11 


105.99 


66 


65.33 


68.27 


71.51 


75.42 


81.09 


85.96 


90.35 


95.63 


99.33 


107.26 


67 


66.33 


69.29 


72.55 


76.50 


82.20 


87.11 


91.52 


96.83 


100.55 


108.53 


68 


67.33 


70.32 


73.60 


77.57 


83.31 


88.25 


92.69 


98.03 


101.78 


109.79 


69 


68.33 


71.34 


74.64 


78.64 


84.42 


89.39 


93.86 


99.23 


103.00 


111.06 


70 


69.33 


72.36 


75.69 


79.71 


85.53 


90.53 


95.02 


100.43 


104.21 


112.32 



Download free eBooks at bookboon.com 



93 



Statistics Tables 



B.4 Student's t distribution (values x with F student (x) = 0.600 etc.) 



df 0.600 0.700 0.800 0.900 0.950 0.975 0.990 0.995 0.999 



1 


0.32 


0.73 


1.38 


3.08 


6.31 


12.71 


31.82 


63.66 


318.31 


2 


0.29 


0.62 


1.06 


1.89 


2.92 


4.30 


6.96 


9.92 


22.33 


3 


0.28 


0.58 


0.98 


1.64 


2.35 


3.18 


4.54 


5.84 


10.2 


4 


0.27 


0.57 


0.94 


1.53 


2.13 


2.78 


3.75 


4.60 


7.17 


5 


0.27 


0.56 


0.92 


1.48 


2.02 


2.57 


3.36 


4.03 


5.89 


6 


0.26 


0.55 


0.91 


1.44 


1.94 


2.45 


3.14 


3.71 


5.21 


7 


0.26 


0.55 


0.90 


1.41 


1.89 


2.36 


3.00 


3.50 


4.79 


8 


0.26 


0.55 


0.89 


1.40 


1.86 


2.31 


2.90 


3.36 


4.50 


9 


0.26 


0.54 


0.88 


1.38 


1.83 


2.26 


2.82 


3.25 


4.30 


10 


0.26 


0.54 


0.88 


1.37 


1.81 


2.23 


2.76 


3.17 


4.14 


11 


0.26 


0.54 


0.88 


1.36 


1.80 


2.20 


2.72 


3.11 


4.02 


12 


0.26 


0.54 


0.87 


1.36 


1.78 


2.18 


2.68 


3.05 


3.93 


13 


0.26 


0.54 


0.87 


1.35 


1.77 


2.16 


2.65 


3.01 


3.85 


14 


0.26 


0.54 


0.87 


1.35 


1.76 


2.14 


2.62 


2.98 


3.79 


15 


0.26 


0.54 


0.87 


1.34 


1.75 


2.13 


2.60 


2.95 


3.73 


16 


0.26 


0.54 


0.86 


1.34 


1.75 


2.12 


2.58 


2.92 


3.69 


17 


0.26 


0.53 


0.86 


1.33 


1.74 


2.11 


2.57 


2.90 


3.65 


18 


0.26 


0.53 


0.86 


1.33 


1.73 


2.10 


2.55 


2.88 


3.61 


19 


0.26 


0.53 


0.86 


1.33 


1.73 


2.09 


2.54 


2.86 


3.58 


20 


0.26 


0.53 


0.86 


1.33 


1.72 


2.09 


2.53 


2.85 


3.55 


21 


0.26 


0.53 


0.86 


1.32 


1.72 


2.08 


2.52 


2.83 


3.53 


22 


0.26 


0.53 


0.86 


1.32 


1.72 


2.07 


2.51 


2.82 


3.50 


23 


0.26 


0.53 


0.86 


1.32 


1.71 


2.07 


2.50 


2.81 


3.48 


24 


0.26 


0.53 


0.86 


1.32 


1.71 


2.06 


2.49 


2.80 


3.47 


25 


0.26 


0.53 


0.86 


1.32 


1.71 


2.06 


2.49 


2.79 


3.45 


26 


0.26 


0.53 


0.86 


1.31 


1.71 


2.06 


2.48 


2.78 


3.43 


27 


0.26 


0.53 


0.86 


1.31 


1.70 


2.05 


2.47 


2.77 


3.42 


28 


0.26 


0.53 


0.85 


1.31 


1.70 


2.05 


2.47 


2.76 


3.41 


29 


0.26 


0.53 


0.85 


1.31 


1.70 


2.05 


2.46 


2.76 


3.40 


30 


0.26 


0.53 


0.85 


1.31 


1.70 


2.04 


2.46 


2.75 


3.39 


35 


0.26 


0.53 


0.85 


1.31 


1.69 


2.03 


2.44 


2.72 


3.34 


40 


0.26 


0.53 


0.85 


1.30 


1.68 


2.02 


2.42 


2.70 


3.31 


50 


0.25 


0.53 


0.85 


1.30 


1.68 


2.01 


2.40 


2.68 


3.26 


00 


0.25 


0.53 


0.85 


1.29 


1.66 


1.98 


2.36 


2.63 


3.17 


oo 


0.25 


0.52 


0.84 


1.28 


1.64 


1.96 


2.33 


2.58 


3.09 
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B.5 Fisher's F distribution (values x with F Fisher (x) = 0.90) 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


39.86 


49.50 


53.59 


55.83 


57.24 


58.20 


58.91 


59.44 


59.86 


60.19 


2 


8.53 


9.00 


9.16 


9.24 


9.29 


9.33 


9.35 


9.37 


9.38 


9.39 


3 


5.54 


5.46 


5.39 


5.34 


5.31 


5.28 


5.27 


5.25 


5.24 


5.23 


4 


4.54 


4.32 


4.19 


4.11 


4.05 


4.01 


3.98 


3.95 


3.94 


3.92 


5 


4.06 


3.78 


3.62 


3.52 


3.45 


3.40 


3.37 


3.34 


3.32 


3.30 


6 


3.78 


3.46 


3.29 


3.18 


3.11 


3.05 


3.01 


2.98 


2.96 


2.94 


7 


3.59 


3.26 


3.07 


2.96 


2.88 


2.83 


2.78 


2.75 


2.72 


2.70 


8 


3.46 


3.11 


2.92 


2.81 


2.73 


2.67 


2.62 


2.59 


2.56 


2.54 


9 


3.36 


3.01 


2.81 


2.69 


2.61 


2.55 


2.51 


2.47 


2.44 


2.42 


10 


3.29 


2.92 


2.73 


2.61 


2.52 


2.46 


2.41 


2.38 


2.35 


2.32 


11 


3.23 


2.86 


2.66 


2.54 


2.45 


2.39 


2.34 


2.30 


2.27 


2.25 


12 


3.18 


2.81 


2.61 


2.48 


2.39 


2.33 


2.28 


2.24 


2.21 


2.19 


13 


3.14 


2.76 


2.56 


2.43 


2.35 


2.28 


2.23 


2.20 


2.16 


2.14 


14 


3.10 


2.73 


2.52 


2.39 


2.31 


2.24 


2.19 


2.15 


2.12 


2.10 


15 


3.07 


2.70 


2.49 


2.36 


2.27 


2.21 


2.16 


2.12 


2.09 


2.06 


16 


3.05 


2.67 


2.46 


2.33 


2.24 


2.18 


2.13 


2.09 


2.06 


2.03 


17 


3.03 


2.64 


2.44 


2.31 


2.22 


2.15 


2.10 


2.06 


2.03 


2.00 


18 


3.02 


2.62 


2.42 


2.29 


2.20 


2.13 


2.08 


2.04 


2.00 


1.98 


19 


3.01 


2.61 


2.40 


2.27 


2.18 


2.11 


2.06 


2.02 


1.98 


1.96 


20 


3.00 


2.59 


2.38 


2.25 


2.16 


2.09 


2.04 


2.00 


1.96 


1.94 


21 


2.98 


2.57 


2.36 


2.23 


2.14 


2.08 


2.02 


1.98 


1.95 


1.92 


22 


2.97 


2.56 


2.35 


2.22 


2.13 


2.06 


2.01 


1.97 


1.93 


1.90 


23 


2.96 


2.55 


2.34 


2.21 


2.11 


2.05 


1.99 


1.95 


1.92 


1.89 


24 


2.95 


2.54 


2.33 


2.19 


2.10 


2.04 


1.98 


1.94 


1.91 


1.88 


25 


2.94 


2.53 


2.32 


2.18 


2.09 


2.02 


1.97 


1.93 


1.89 


1.87 


26 


2.93 


2.52 


2.31 


2.17 


2.08 


2.01 


1.96 


1.92 


1.88 


1.86 


27 


2.92 


2.51 


2.30 


2.17 


2.07 


2.00 


1.95 


1.91 


1.87 


1.85 


28 


2.92 


2.50 


2.29 


2.16 


2.06 


2.00 


1.94 


1.90 


1.87 


1.84 


29 


2.91 


2.50 


2.28 


2.15 


2.06 


1.99 


1.93 


1.89 


1.86 


1.83 


30 


2.90 


2.49 


2.28 


2.14 


2.05 


1.98 


1.93 


1.88 


1.85 


1.82 


31 


2.90 


2.48 


2.27 


2.14 


2.04 


1.97 


1.92 


1.88 


1.84 


1.81 


32 


2.89 


2.48 


2.26 


2.13 


2.04 


1.97 


1.91 


1.87 


0.84 


1.81 


33 


2.89 


2.47 


2.26 


2.12 


2.03 


1.96 


1.91 


1.86 


1.83 


1.80 


34 


2.88 


2.47 


2.25 


2.12 


2.02 


1.96 


1.90 


1.86 


1.82 


1.79 


35 


2.88 


2.46 


2.25 


2.11 


2.02 


1.95 


1.90 


1.85 


1.82 


1.79 
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B.6 Fisher's F distribution (values x with F Fisher (x) = 0.95) 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


161.45 


199.50 


215.71 


224.58 


230.16 


233.99 


236.77 


238.88 


240.54 


241.88 


2 


18.51 


19.00 


19.16 


19.25 


19.30 


19.33 


19.35 


19.37 


19.38 


19.40 


3 


10.13 


9.55 


9.28 


9.12 


9.01 


8.94 


8.89 


8.85 


8.81 


8.79 


4 


7.71 


6.94 


6.59 


6.39 


6.26 


6.16 


6.09 


6.04 


6.00 


5.96 


5 


6.61 


5.79 


5.41 


5.19 


5.05 


4.95 


4.88 


4.82 


4.77 


4.74 


6 


5.99 


5.14 


4.76 


4.53 


4.39 


4.28 


4.21 


4.15 


4.10 


4.06 


7 


5.59 


4.74 


4.35 


4.12 


3.97 


3.87 


3.79 


3.73 


3.68 


3.64 


8 


5.32 


4.46 


4.07 


3.84 


3.69 


3.58 


3.50 


3.44 


3.39 


3.35 


9 


5.12 


4.26 


3.86 


3.63 


3.48 


3.37 


3.29 


3.23 


3.18 


3.14 


10 


4.96 


4.10 


3.71 


3.48 


3.33 


3.22 


3.14 


3.07 


3.02 


2.98 


11 


4.84 


3.98 


3.59 


3.36 


3.20 


3.09 


3.01 


2.95 


2.90 


2.85 


12 


4.75 


3.89 


3.49 


3.26 


3.11 


3.00 


2.91 


2.85 


2.80 


2.75 


13 


4.67 


3.81 


3.41 


3.18 


3.03 


2.92 


2.83 


2.77 


2.71 


2.67 


14 


4.60 


3.74 


3.34 


3.11 


2.96 


2.85 


2.76 


2.70 


2.65 


2.60 


15 


4.54 


3.68 


3.29 


3.06 


2.90 


2.79 


2.71 


2.64 


2.59 


2.54 


16 


4.49 


3.63 


3.24 


3.01 


2.85 


2.74 


2.66 


2.59 


2.54 


2.49 


17 


4.45 


3.59 


3.20 


2.96 


2.81 


2.70 


2.61 


2.55 


2.49 


2.45 


18 


4.43 


3.55 


3.16 


2.93 


2.77 


2.66 


2.58 


2.51 


2.46 


2.41 


19 


4.41 


3.52 


3.13 


2.90 


2.74 


2.63 


2.54 


2.48 


2.42 


2.38 


20 


4.38 


3.49 


3.10 


2.87 


2.71 


2.60 


2.51 


2.45 


2.39 


2.35 


21 


4.35 


3.47 


3.07 


2.84 


2.68 


2.57 


2.49 


2.42 


2.37 


2.32 


22 


4.33 


3.44 


3.05 


2.82 


2.66 


2.55 


2.46 


2.40 


2.34 


2.30 


23 


4.31 


3.42 


3.03 


2.80 


2.64 


2.53 


2.44 


2.37 


2.32 


2.27 


24 


4.29 


3.40 


3.01 


2.78 


2.62 


2.51 


2.42 


2.36 


4.62 


2.25 


25 


4.27 


3.39 


2.99 


2.76 


2.60 


2.49 


2.40 


2.34 


2.28 


2.24 


26 


4.25 


3.37 


2.98 


2.74 


2.59 


2.47 


2.39 


2.32 


2.27 


2.22 


27 


4.24 


3.35 


2.96 


2.73 


2.57 


2.46 


2.37 


2.31 


2.25 


2.20 


28 


4.22 


3.34 


2.95 


2.71 


2.56 


2.45 


2.36 


2.29 


2.24 


2.19 


29 


4.21 


3.33 


2.93 


2.70 


2.55 


2.43 


2.35 


2.28 


2.22 


2.18 


30 


4.20 


3.32 


2.92 


2.69 


2.53 


2.42 


2.33 


2.27 


2.21 


2.16 


31 


4.18 


3.30 


2.91 


2.68 


2.52 


2.41 


2.32 


2.25 


2.20 


2.15 


32 


4.17 


3.29 


2.90 


2.67 


2.51 


2.40 


2.31 


2.24 


2.19 


2.14 


33 


4.16 


3.28 


2.89 


2.66 


2.50 


2.39 


2.30 


2.23 


2.18 


2.13 


34 


4.15 


3.28 


2.88 


2.65 


2.49 


2.38 


2.29 


2.23 


2.17 


2.12 


35 


4.15 


3.27 


2.87 


2.64 


2.49 


2.37 


2.29 


2.22 


2.16 


2.11 
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B.7 Fisher's F distribution (values x with F Fisher (x) = 0.99) 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


4052 


5000 


5403 


5625 


5764 


5859 


5928 


5981 


6022 


6056 


2 


98.50 


99.00 


99.17 


99.25 


99.30 


99.33 


99.36 


99.37 


99.39 


99.40 


3 


34.12 


30.82 


29.46 


28.71 


28.24 


27.91 


27.67 


27.49 


27.35 


27.23 


4 


21.20 


18.00 


16.69 


15.98 


15.52 


15.21 


14.98 


14.80 


14.66 


14.55 


5 


16.26 


13.27 


12.06 


11.39 


10.97 


10.67 


10.46 


10.29 


10.16 


10.05 


6 


13.75 


10.92 


9.78 


9.15 


8.75 


8.47 


8.26 


8.10 


7.98 


7.87 


7 


12.25 


9.55 


8.45 


7.85 


7.46 


7.19 


6.99 


6.84 


6.72 


6.62 


8 


11.26 


8.65 


7.59 


7.01 


6.63 


6.37 


6.18 


6.03 


5.91 


5.81 


9 


10.56 


8.02 


6.99 


6.42 


6.06 


5.80 


5.61 


5.47 


5.35 


5.26 


10 


10.04 


7.56 


6.55 


5.99 


5.64 


5.39 


5.20 


5.06 


4.94 


4.85 


11 


9.65 


7.21 


6.22 


5.67 


5.32 


5.07 


4.89 


4.74 


4.63 


4.54 


12 


9.33 


6.93 


5.95 


5.41 


5.06 


4.82 


4.64 


4.50 


4.39 


4.30 


13 


9.07 


6.70 


5.74 


5.21 


4.86 


4.62 


4.44 


4.30 


4.19 


4.10 


14 


8.86 


6.51 


5.56 


5.04 


4.69 


4.46 


4.28 


4.14 


4.03 


3.94 


15 


8.68 


6.36 


5.42 


4.89 


4.56 


4.32 


4.14 


4.00 


3.89 


3.80 


16 


8.53 


6.23 


5.29 


4.77 


4.44 


4.20 


4.03 


3.89 


3.78 


3.69 


17 


8.40 


6.11 


5.18 


4.67 


4.34 


4.10 


3.93 


3.79 


3.68 


3.59 


18 


8.30 


6.01 


5.09 


4.58 


4.25 


4.01 


3.84 


3.71 


3.60 


3.51 


19 


8.22 


5.93 


5.01 


4.50 


4.17 


3.94 


3.77 


3.63 


3.52 


3.43 


20 


8.13 


5.85 


4.94 


4.43 


4.10 


3.87 


3.70 


3.56 


3.46 


3.37 


21 


8.05 


5.78 


4.87 


4.37 


4.04 


3.81 


3.64 


3.51 


3.40 


3.31 


22 


7.98 


5.72 


4.82 


4.31 


3.99 


3.76 


3.59 


3.45 


3.35 


3.26 


23 


7.91 


5.66 


4.76 


4.26 


3.94 


3.71 


3.54 


3.41 


3.30 


3.21 


24 


7.85 


5.61 


4.72 


4.22 


3.90 


3.67 


3.50 


3.36 


3.26 


3.17 


25 


7.80 


5.57 


4.68 


4.18 


3.85 


3.63 


3.46 


3.32 


3.22 


3.13 


26 


7.75 


5.53 


4.64 


4.14 


3.82 


3.59 


3.42 


3.29 


3.18 


3.09 


27 


7.71 


5.49 


4.60 


4.11 


3.78 


3.56 


3.39 


3.26 


3.15 


3.06 


28 


7.67 


5.45 


4.57 


4.07 


3.75 


3.53 


3.36 


3.23 


3.12 


3.03 


29 


7.63 


5.42 


4.54 


4.04 


3.73 


3.50 


3.33 


3.20 


3.09 


3.00 


30 


7.59 


5.39 


4.51 


4.02 


3.70 


3.47 


3.30 


3.17 


3.07 


2.98 


31 


7.56 


5.36 


4.48 


3.99 


3.67 


3.45 


3.28 


3.15 


3.04 


2.96 


32 


7.53 


5.34 


4.46 


3.97 


3.65 


3.43 


3.26 


3.13 


3.02 


2.93 


33 


7.50 


5.31 


4.44 


3.95 


3.63 


3.41 


3.24 


3.11 


3.00 


2.91 


34 


7.47 


5.29 


4.42 


3.93 


3.61 


3.39 


3.22 


3.09 


2.98 


2.89 


35 


7.45 


5.27 


4.40 


3.91 


3.59 


3.37 


3.20 


3.07 


2.96 


2.88 
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B.8 Wilcoxon's test for one set of observations 



n 


0.005 


0.010 


0.025 


0.050 


5 


- 


- 


- 





6 


- 


- 





2 


7 


- 





2 


3 


8 





1 


3 


5 


9 


1 


3 


5 


8 


10 


3 


5 


8 


10 


11 


5 


7 


10 


13 


12 


7 


9 


13 


17 


13 


9 


12 


17 


21 


14 


12 


15 


21 


25 


15 


15 


19 


25 


30 


16 


19 


23 


29 


35 


17 


23 


27 


34 


41 


18 


27 


32 


40 


47 


19 


32 


37 


46 


53 


20 


37 


43 


52 


60 


21 


42 


49 


58 


67 


22 


48 


55 


65 


75 


23 


54 


62 


73 


83 


24 


61 


69 


81 


91 


25 


68 


76 


89 


100 


26 


75 


84 


98 


110 


27 


83 


92 


107 


119 



n 


0.005 


0.010 


0.025 


0.050 


28 


91 


101 


116 


130 


29 


100 


110 


126 


140 


30 


109 


120 


137 


151 


31 


118 


130 


147 


163 


32 


128 


140 


159 


175 


33 


138 


151 


170 


187 


34 


148 


162 


182 


200 


35 


159 


173 


195 


213 


36 


171 


185 


208 


227 


37 


182 


198 


221 


241 


38 


194 


211 


235 


256 


39 


207 


224 


249 


271 


40 


220 


238 


264 


286 


41 


233 


252 


279 


302 


42 


247 


266 


294 


319 


43 


261 


281 


310 


336 


44 


276 


296 


327 


353 


45 


291 


312 


343 


371 


46 


307 


328 


361 


389 


47 


322 


345 


378 


407 


48 


339 


362 


396 


426 


49 


355 


379 


415 


446 


50 


373 


397 


434 


466 
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B.9 Wilcoxon's test for two sets of observations, a = 

23456789 



m 



10 11 12 13 14 15 



n= 1 















































2 


2 


2 


2 


2 


3 


3 


3 


4 


4 


4 


4 


5 


5 


6 


6 


3 


5 


5 


6 


6 


7 


8 


8 


9 


10 


10 


11 


11 


12 


13 


13 


4 


9 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


5 


14 


15 


16 


17 


19 


20 


21 


23 


24 


26 


27 


28 


30 


31 


33 


6 


20 


21 


23 


24 


26 


28 


29 


31 


33 


35 


37 


38 


40 


42 


44 


7 


27 


28 


30 


32 


34 


36 


39 


41 


43 


45 


47 


49 


52 


54 


56 


8 


35 


37 


39 


41 


44 


46 


49 


51 


54 


56 


59 


62 


64 


67 


69 


9 


44 


46 


49 


51 


54 


57 


60 


63 


66 


69 


72 


75 


78 


81 


84 


10 


54 


56 


59 


62 


66 


69 


72 


75 


79 


82 


86 


89 


92 


96 


99 


11 


65 


67 


71 


74 


78 


82 


85 


89 


93 


97 


100 


104 


108 


112 


116 


12 


77 


80 


83 


87 


91 


95 


99 


104 


108 


112 


116 


120 


125 


129 


133 


13 


90 


93 


97 


101 


106 


110 


115 


119 


124 


128 


133 


138 


142 


147 


152 


14 


104 


108 


112 


116 


121 


126 


131 


136 


141 


146 


151 


156 


161 


166 


171 


15 


119 


123 


127 


132 


138 


143 


148 


153 


159 


164 


170 


175 


181 


186 


192 


16 


135 


139 


144 


150 


155 


161 


166 


172 


178 


184 


190 


196 


201 


207 


213 


17 


152 


156 


162 


168 


173 


179 


186 


192 


198 


204 


210 


217 


223 


230 


236 


18 


170 


175 


180 


187 


193 


199 


206 


212 


219 


226 


232 


239 


246 


253 


259 


19 


190 


194 


200 


207 


213 


220 


227 


234 


241 


248 


255 


262 


270 


277 


284 


20 


210 


214 


221 


228 


235 


242 


249 


257 


264 


272 


279 


287 


294 


302 


310 


21 


231 


236 


242 


250 


257 


265 


272 


280 


288 


296 


304 


312 


320 


328 


336 


22 


253 


258 


265 


273 


281 


289 


297 


305 


313 


321 


330 


338 


347 


355 


364 


23 


276 


281 


289 


297 


305 


313 


322 


330 


339 


348 


357 


366 


374 


383 


392 


24 


300 


306 


313 


322 


330 


339 


348 


357 


366 


375 


385 


394 


403 


413 


422 


25 


325 


331 


339 


348 


357 


366 


375 


385 


394 


404 


414 


423 


433 


443 


453 
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C Explanation of symbols 



A, B,C 


events 


n 


sample space 


p 


probability function, significance probability 


P(A\B) 


conditional probability of A given B 


n, u 


intersection, union 


A, V 


and, or 


ACQ 


A is a subset of 


weU 


uj belongs to Q 


CM 


complement of the set A 


A\B 


difference of the sets A and B ("A minus £>") 


/:fi^R 


/ is a map from ft into R 


:= 


equals by definition 


\x\ 


absolute value of x (e.g. —2=2) 


N,Z,R 


the set of natural, integral, real numbers 





the empty set 


[0,oo [ 


the interval {x e R \x > 0} 


X,Y 


random variables 


E(X) 


the expected value of X 


var(X) 


the variance of X 


Cov(X, Y) 


the covariance of X and Y 


li 


expected value 


a 2 


variance 


a 


standard deviation 


Bin 


binomial distribution 


Pois 


Poisson distribution 


Geo 


geometrical distribution 


HG 


hypergeometrical distribution 


Mult 


multinomial distribution 


NB 


negative binomial distribution 


Exp 


exponential distribution 


N 


normal distribution 


s 2 


empirical variance 


s 


empirical standard deviation 


F(x) 


distribution function 


fix) 


density function 


<&(x) 


distribution function of standard normal distribution 


(f(x) 


density function of standard normal distribution 


n 


number of observations or tries 


A 


intensity (in a Poisson process) 


R 2 


determination coefficient 
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p correlation coefficient 

x, y mean value 

df number of degrees of freedom 
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Index 

accept, 29 

addition formula, 24 

addition formula for normal distribution, 50 

alternative hypothesis, 29 

analysis of variance, 60 

ANOVA, 60 

automobiles, 36 

auxiliary hypothesis, 59 

binomial coefficient, 16 
binomial distribution, 30 
blackballs, 41 
Bortkiewicz, Ladislaus von, 69 

cabinet, 42 

Central Limit Theorem, 26 

Chebyshev's inequality, 26 

chi- squared, 63 

chi-squared distribution, 50 

city council, 42 

conditional probability, 12 

contingency table, 70 

continuous random variable, 20 

correlation coefficient, 24 

covariance, 24 

cumulative probabilities, 32 

declaring variable, 79 
density, 20 
density function, 20 
determination coefficient, 81 
discrepancy, 64 
discrete random variable, 19 
disjoint events, 12 
distribution function, 18 
distribution-free test, 74 
division into possible causes, 13 

empirical correlation coefficient, 29 
empirical covariance, 29 
empirical standard deviation, 28 



empirical variance, 28 
equality of distribution, 63 
errors (of type I and II), 30 
event, 12 

expected number, 64 
expected value, 21, 24 
exponential distribution, 45 

F distribution, 52 
Fisher's exact test, 73 
Fisher's F, 52 

Fisher-Behrens problem, 57 
French Open, 13 

generalized binomial coefficients, 17 
geometrical distribution, 39 
Gesetz der kleinen Zahlen, 69 
goodness of fit, 63 
Green Party, 34 

hypergeometrical distribution, 41 

inclusion-exclusion, 14 
independent events, 14 
independent random variables, 20 
intensity, 35, 45 
interquartile range, 28 

Law of Large Numbers, 26 
linear regression, 79 
lower quartile, 28 

marginal density, 21 

mean, 28 

mean value, 28 

median, 27 

moment of extraordinary beauty, 27 

multinomial coefficients, 17 

multinomial distribution, 43 

multiplication formula, 24 

negative binomial distribution, 44 
non-parametric test, 74 
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normal distribution, 46 
normed sum, 26 
null hypothesis, 29 

observed number, 64 

parameter, 30 
Pascal's triangle, 17 
Pearson's test, 63 
pin diagram, 19 
point probability, 19 
Poisson distribution, 35 
pooled variance, 57 
predicted value, 80 
probability function, 12 
probability of failure, 30 
probability of success, 30 
probability space, 12 
Prussian cavalry, 68 

quartile, 28 

random variables, 18 
random vector, 21 
rank, 74 
red balls, 41 
regression, 79 
reject, 29 

remainder term, 79 
residual, 80 
response variable, 79 

sample point, 12 

sample space, 12 

school class, 15 

significance level, 29 

significance probability, 29 

simple linear regression, 79 

simultaneous density, 21 

simultaneous distribution function, 21 

Skoda, 36 

spontaneous event, 35, 45 

standard deviation, 22 

standard normal distribution, 47 



standardized residuals, 65, 71 
strength, 30 
Student's t, 51 

t distribution, 51 
test, 29 

throw of a dice, 12 
toss of a coin, 27 
two-times-two table, 73 
type I and II, 30 

unbiased estimator, 33 
upper quartile, 28 
urn, 41 

variance, 22, 24 

variance between samples, 60 

variance within samples, 60 

Wilcoxon's test, 74 



103 



